Ultrasonic vibrations on a window in a lidar system

ABSTRACT

To clear the line-of-sight (LOS) on a window in a lidar system within a vehicle, ultrasonic transducers are physically attached to the interior surface of the window to remove debris. When debris is detected at the window, the ultrasonic transducers are directed to vibrate at an ultrasonic frequency thereby generating an ultrasonic wave which causes the debris to bounce off the window. In some scenarios, the ultrasonic transducers vibrate with a particular phase shift to generate a travelling wave that causes the debris to sweep across the window. Additionally, windshield wipers, spray from a nozzle, and/or a hydrophobic or oleophobic coating may be used in combination with the ultrasonic transducers to remove the debris from the window.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to provisional U.S. Application Ser.No. 62/478,281, filed on Mar. 29, 2017, entitled “Ultrasonic Vibrationson a Window in a Lidar System” the entire disclosure of which is herebyexpressly incorporated by reference herein.

FIELD OF TECHNOLOGY

This disclosure generally relates to lidar systems and, moreparticularly, to detecting debris on a window of a lidar system andutilizing an ultrasonic transducer to clear the debris from the window.

BACKGROUND

The background description provided herein is for the purpose ofgenerally presenting the context of the disclosure. Work of thepresently named inventors, to the extent it is described in thisbackground section, as well as aspects of the description that may nototherwise qualify as prior art at the time of filing, are neitherexpressly nor impliedly admitted as prior art against the presentdisclosure.

Light detection and ranging (lidar) is a technology that can be used tomeasure distances to remote targets. Typically, a lidar system includesa light source and an optical receiver. The light source can be, forexample, a laser which emits light having a particular operatingwavelength. The operating wavelength of a lidar system may lie, forexample, in the infrared, visible, or ultraviolet portions of theelectromagnetic spectrum. The light source emits light toward a targetwhich then scatters the light. Some of the scattered light is receivedback at the receiver. The system determines the distance to the targetbased on one or more characteristics associated with the returned light.For example, the system may determine the distance to the target basedon the time of flight of a returned light pulse.

SUMMARY

One example embodiment of the techniques of this disclosure is a lidarsystem including a light source configured to emit light pulses, ascanner configured to scan a field of regard of the lidar system,including direct the light pulses at different angles relative to aforward-facing direction toward different points within the field ofregard to illuminate a field of view of the light source, and a receiverconfigured to detect light from some of the light pulses scattered byone or more remote targets to generate respective pixels. The lidarsystem further includes a window having an exterior-facing surface, thewindow arranged in front of the light source, the scanner, and thereceiver, where the emitted light pulses are transmitted through thewindow toward the one or more remote targets and light from the somelight pulses is scattered by the one or more remote targets, transmittedthrough the window, and detected at the receiver. Additionally, thelidar system includes one or more ultrasonic transducers attached to thewindow and configured to cause vibration at the window to remove debrisfrom the exterior-facing surface of the window.

Another embodiment of the techniques of this disclosure is a method forremoving debris from a window of a lidar system. The method includesemitting light pulses by a light source in a lidar system, scanning, bya scanner in the lidar system, a field of regard of the lidar system,including directing the light pulses toward different points within thefield of regard to illuminate a field of view of the light source, anddetecting, by a receiver in the lidar system, light from some of thelight pulses scattered by one or more remote targets to generaterespective pixels. The method further includes detecting debris on awindow having an exterior-facing surface arranged in front of the lightsource, the scanner, and the receiver, where the emitted light pulsesare transmitted through the window toward the one or more remote targetsand light from the some light pulses is scattered by the one or moreremote targets, transmitted through the window, and detected at thereceiver, and causing one or more ultrasonic transducers attached to thewindow to vibrate in response to detecting the debris on the window toremove the debris from the exterior-facing surface of the window.

Yet another embodiment of the techniques of this disclosure is acontroller in a lidar system. The controller includes one or moreprocessors and a non-transitory computer-readable memory coupled to theone or more processors and storing instructions thereon. Theinstructions, when executed by the one or more processors, cause thecontroller to provide a control signal to a light source to emit lightpulses and detect debris on a window having an exterior-facing surfacearranged in front of the light source, a scanner, and a receiver, wherethe emitted light pulses are transmitted through the window toward oneor more remote targets and light from the some light pulses is scatteredby the one or more remote targets, transmitted through the window, anddetected at the receiver. The instructions further cause the controllerto provide a control signal to one or more ultrasonic transducersattached to the window to vibrate to remove debris from theexterior-facing surface of the window.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example light detection and ranging(lidar) system in which the techniques of this disclosure can beimplemented;

FIG. 2 illustrates in more detail several components that can operate inthe system of FIG. 1;

FIG. 3 illustrates an example configuration in which the components ofFIG. 1 scan a 360-degree field of regard through a window in a rotatinghousing;

FIG. 4 illustrates another configuration in which the components of FIG.1 scan a 360-degree field of regard through a substantially transparentstationary housing;

FIG. 5 illustrates an example scan pattern which the lidar system ofFIG. 1 can produce when identifying targets within a field of regard;

FIG. 6 illustrates an example scan pattern which the lidar system ofFIG. 1 can produce when identifying targets within a field of regardusing multiple channels;

FIG. 7 schematically illustrates fields of view (FOVs) of a light sourceand a detector that can operate in the lidar system of FIG. 1;

FIG. 8 illustrates an example configuration of the lidar system of FIG.1 or another suitable lidar system, in which a laser is disposed awayfrom sensor components;

FIG. 9 illustrates an example vehicle in which the lidar system of FIG.1 can operate; a

FIG. 10 illustrates an example InGaAs avalanche photodiode which canoperate in the lidar system of FIG. 1;

FIG. 11 illustrates an example photodiode coupled to a pulse-detectioncircuit, which can operate in the lidar system of FIG. 1;

FIG. 12 illustrates a flow diagram of an example method for detectingand removing debris on a window of a lidar system;

FIG. 13 illustrates a flow diagram of an example method for verifyingthat debris has been removed from a window of a lidar system; and

FIG. 14 illustrates a flow diagram of an example method for adjustingcharacteristics of a wave created by ultrasonic transducers attached toa window in a lidar system based on the amount of debris on the window.

DETAILED DESCRIPTION Overview

Generally speaking, a light source (e.g., a fiber laser or a diodelaser) in a lidar system transmits light pulses through a window of thelidar system. The light pulses are directed by a scanner across a fieldof regard of the lidar system, and light from the light pulses isscattered by one or more remote targets, transmitted through the window,and detected at a receiver in the lidar system. When the window hasdebris on its interior or exterior surface, emitted light pulses may beprevented from being transmitted across the field of regard, and returnlight pulses may be prevented from being detected at the receiver.Depending on the type of particles, the debris may block light pulsescompletely or partially. To clear the line-of-sight (LOS) on the window,one or several ultrasonic transducers are physically attached to thewindow. When debris is detected, the ultrasonic transducers operate tocreate a wave at an ultrasonic frequency at the window. The ultrasonicwaves generally facilitate removal of water and other debris off thesurface of the window by triggering cavitation in the fluid present inthe debris, for example.

To detect the presence of debris on the window, one or more dedicatedsensors, such as a rain sensor, an air pressure sensor, and/or anothersensor capable of detecting conditions indicative of current weather,may be included in the lidar system. The rain sensor may be an opticalrain sensor operating in the infrared range, for example. In someimplementations, a controller in the lidar system is communicativelycoupled to the one or more sensors to receive an indication that thereis debris on the window. The controller then provides a control signalto the ultrasonic transducers to produce ultrasonic vibrations.

In some implementations, the lidar system does not utilize dedicatedrain sensors (or other dedicated sensors) to detect debris, and insteadrelies on properties of received light pulses to determine that debrismay be present. Depending on the composition, the debris may reflectmuch of the light pulse, effectively creating an early return pulse, orconversely, absorb much of the light pulse. The lidar system in theseimplementations may include a controller coupled to the receiver. Whenthe receiver does not detect a return light pulse within the sameangular region within the field of regard during a certain time periodfor a certain number of scan frames while the vehicle is moving, or whenthe light pulse returns too early, the controller may determine that theview is probably blocked at that particular orientation by debris.

Further, the lidar system in some implementations uses signals fromsensors operating in the vehicle. For example, certain vehicles areequipped with rain sensors, and the lidar system may be configured toreceive a signal from a rain sensor operating in the vehicle (directlyor via an intermediate vehicle controller) and infer that the conditionof the window of the lidar system is similar to the condition of thewindshield of the car. Still further, the lidar system can in some casesrely on weather data received from a network server to determine thelikely condition of the window of the lidar system. For example, acontroller operating in the vehicle can receive an indication from aremote service that that it is raining in the region through which thevehicle currently is traveling. The controller accordingly may provide acorresponding indication to the lidar system, which in response mayactivate ultrasonic transducers.

Additionally or alternatively, the lidar system may conduct a diagnosticroutine to detect whether any portions of the field of regard areobscured. Upon identifying such areas, the lidar system may initiate aremedial procedure that includes operating the ultrasonic transducers.The lidar system then may execute the diagnostic routine again to seewhether the remedial action had any effect on the field of regard. Forexample, the lidar system may receive an indication from another source,such as a camera mounted on the vehicle, that the vehicle currently isnot facing any objects within the maximum distance at which the lidarsystem operates, in the entire field of regard or a specified portion ofthe field of regard. The lidar system accordingly may emit pulses anddetermine whether any of the pulses are reflected back.

The lidar system in some cases may activate the ultrasonic transducersaccording to a certain schedule. For example, the lidar system mayactivate the ultrasonic transducers every N seconds to clear the debris,if any. The lidar system in these implementations does not depend onsensor readings to determine when an attempt to clear the window shouldbe initiated.

The lidar system also may generate one or more indications for theoperator, passenger, or driver of the vehicle to indicate that debris islikely present on the surface of the window. The lidar system maygenerate these indications in addition to attempting to clear the debriswith ultrasonic signals or as an alternative to this remedial action. Insome cases, the lidar system may determine the likely presence ofdebris, attempt to clear the debris using ultrasonic signals, determinewhether the debris is still being observed, and generate an indicationfor the user only if the remedial action appears to have beenunsuccessful.

An example lidar system in which these techniques can be implemented isconsidered next with reference to FIGS. 1-4, followed by a discussion ofthe techniques which the lidar system can implement to scan a field ofregard and generate individual pixels (FIGS. 5-7). An exampleimplementation in a vehicle is then discussed with reference to FIGS. 8and 9. Then, an example photo detector and an example pulse-detectioncircuit are discussed with reference to FIGS. 10 and 11.

System Overview

FIG. 1 illustrates an example light detection and ranging (lidar) system100. The lidar system 100 may be referred to as a laser ranging system,a laser radar system, a LIDAR system, a lidar sensor, or a laserdetection and ranging (LADAR or ladar) system. The lidar system 100 mayinclude a light source 110, a mirror 115, a scanner 120, a receiver 140,and a controller 150. The light source 110 may be, for example, a laserwhich emits light having a particular operating wavelength in theinfrared, visible, or ultraviolet portions of the electromagneticspectrum. As a more specific example, the light source 110 may include alaser with an operating wavelength between approximately 1.2 μm and 1.7μm.

In operation, the light source 110 emits an output beam of light 125which may be continuous-wave, pulsed, or modulated in any suitablemanner for a given application. The output beam of light 125 is directeddownrange toward a remote target 130 located a distance D from the lidarsystem 100 and at least partially contained within a field of regard ofthe system 100. Depending on the scenario and/or the implementation ofthe lidar system 100, D can be between 1 m and 1 km, for example.

Once the output beam 125 reaches the downrange target 130, the target130 may scatter or, in some cases, reflect at least a portion of lightfrom the output beam 125, and some of the scattered or reflected lightmay return toward the lidar system 100. In the example of FIG. 1, thescattered or reflected light is represented by input beam 135, whichpasses through the scanner 120, which may be referred to as a beamscanner, optical scanner, or laser scanner. The input beam 135 passesthrough the scanner 120 to the mirror 115, which may be referred to asan overlap mirror, superposition mirror, or beam-combiner mirror. Themirror 115 in turn directs the input beam 135 to the receiver 140. Theinput 135 may contain only a relatively small fraction of the light fromthe output beam 125. For example, the ratio of average power, peakpower, or pulse energy of the input beam 135 to average power, peakpower, or pulse energy of the output beam 125 may be approximately 10⁻¹,10⁻², 10⁻³, 10⁻⁴, 10⁻⁵, 10⁻⁶, 10⁻⁷, 10⁻⁸, 10⁻⁹, 10⁻¹⁰, 10⁻¹¹, or 10⁻¹².As another example, if a pulse of the output beam 125 has a pulse energyof 1 microjoule (μJ), then the pulse energy of a corresponding pulse ofthe input beam 135 may have a pulse energy of approximately 10nanojoules (nJ), 1 nJ, 100 picojoules (pJ), 10 pJ, 1 pJ, 100 femtojoules(fJ), 10 fJ, 1 fJ, 100 attojoules (aJ), 10 aJ, or 1 aJ.

The output beam 125 may be referred to as a laser beam, light beam,optical beam, emitted beam, or just beam; and the input beam 135 may bereferred to as a return beam, received beam, return light, receivedlight, input light, scattered light, or reflected light. As used herein,scattered light may refer to light that is scattered or reflected by thetarget 130. The input beam 135 may include light from the output beam125 that is scattered by the target 130, light from the output beam 125that is reflected by the target 130, or a combination of scattered andreflected light from target 130.

The operating wavelength of a lidar system 100 may lie, for example, inthe infrared, visible, or ultraviolet portions of the electromagneticspectrum. The Sun also produces light in these wavelength ranges, andthus sunlight can act as background noise which can obscure signal lightdetected by the lidar system 100. This solar background noise can resultin false-positive detections or can otherwise corrupt measurements ofthe lidar system 100, especially when the receiver 140 includes SPADdetectors (which can be highly sensitive).

Generally speaking, the light from the Sun that passes through theEarth's atmosphere and reaches a terrestrial-based lidar system such asthe system 100 can establish an optical background noise floor for thissystem. Thus, in order for a signal from the lidar system 100 to bedetectable, the signal must rise above the background noise floor. It isgenerally possible to increase the signal-to-noise (SNR) ratio of thelidar system 100 by raising the power level of the output beam 125, butin some situations it may be desirable to keep the power level of theoutput beam 125 relatively low. For example, increasing transmit powerlevels of the output beam 125 can result in the lidar system 100 notbeing eye-safe.

In some implementations, the lidar system 100 operates at one or morewavelengths between approximately 1400 nm and approximately 1600 nm. Forexample, the light source 110 may produce light at approximately 1550nm.

In some implementations, the lidar system 100 operates at frequencies atwhich atmospheric absorption is relatively low. For example, the lidarsystem 100 can operate at wavelengths in the approximate ranges from 980nm to 1110 nm or from 1165 nm to 1400 nm.

In other implementations, the lidar system 100 operates at frequenciesat which atmospheric absorption is high. For example, the lidar system100 can operate at wavelengths in the approximate ranges from 930 nm to980 nm, from 1100 nm to 1165 nm, or from 1400 nm to 1460 nm.

According to some implementations, the lidar system 100 can include aneye-safe laser, or the lidar system 100 can be classified as an eye-safelaser system or laser product. An eye-safe laser, laser system, or laserproduct may refer to a system with an emission wavelength, averagepower, peak power, peak intensity, pulse energy, beam size, beamdivergence, exposure time, or scanned output beam such that emittedlight from the system presents little or no possibility of causingdamage to a person's eyes. For example, the light source 110 or lidarsystem 100 may be classified as a Class 1 laser product (as specified bythe 60825-1 standard of the International Electrotechnical Commission(IEC)) or a Class I laser product (as specified by Title 21, Section1040.10 of the United States Code of Federal Regulations (CFR)) that issafe under all conditions of normal use. In some implementations, thelidar system 100 may be classified as an eye-safe laser product (e.g.,with a Class 1 or Class I classification) configured to operate at anysuitable wavelength between approximately 1400 nm and approximately 2100nm. In some implementations, the light source 110 may include a laserwith an operating wavelength between approximately 1400 nm andapproximately 1600 nm, and the lidar system 100 may be operated in aneye-safe manner. In some implementations, the light source 110 or thelidar system 100 may be an eye-safe laser product that includes ascanned laser with an operating wavelength between approximately 1530 nmand approximately 1560 nm. In some implementations, the lidar system 100may be a Class 1 or Class I laser product that includes a fiber laser orsolid-state laser with an operating wavelength between approximately1400 nm and approximately 1600 nm.

The receiver 140 may receive or detect photons from the input beam 135and generate one or more representative signals. For example, thereceiver 140 may generate an output electrical signal 145 that isrepresentative of the input beam 135. The receiver may send theelectrical signal 145 to the controller 150. Depending on theimplementation, the controller 150 may include one or more processors,an application-specific integrated circuit (ASIC), a field-programmablegate array (FPGA), and/or other suitable circuitry configured to analyzeone or more characteristics of the electrical signal 145 to determineone or more characteristics of the target 130, such as its distancedownrange from the lidar system 100. More particularly, the controller150 may analyze the time of flight or phase modulation for the beam oflight 125 transmitted by the light source 110. If the lidar system 100measures a time of flight of T (e.g., T represents a round-trip time offlight for an emitted pulse of light to travel from the lidar system 100to the target 130 and back to the lidar system 100), then the distance Dfrom the target 130 to the lidar system 100 may be expressed as D=c·T/2,where c is the speed of light (approximately 3.0×10⁸ m/s).

As a more specific example, if the lidar system 100 measures the time offlight to be T=300 ns, then the lidar system 100 can determine thedistance from the target 130 to the lidar system 100 to be approximatelyD=45.0 m. As another example, the lidar system 100 measures the time offlight to be T=1.33 μs and accordingly determines that the distance fromthe target 130 to the lidar system 100 is approximately D=199.5 m. Thedistance D from lidar system 100 to the target 130 may be referred to asa distance, depth, or range of the target 130. As used herein, the speedof light c refers to the speed of light in any suitable medium, such asfor example in air, water, or vacuum. The speed of light in vacuum isapproximately 2.9979×10⁸ m/s, and the speed of light in air (which has arefractive index of approximately 1.0003) is approximately 2.9970×10⁸m/s.

The target 130 may be located a distance D from the lidar system 100that is less than or equal to a maximum range R_(MAX) of the lidarsystem 100. The maximum range R_(MAX) (which also may be referred to asa maximum distance) of a lidar system 100 may correspond to the maximumdistance over which the lidar system 100 is configured to sense oridentify targets that appear in a field of regard of the lidar system100. The maximum range of lidar system 100 may be any suitable distance,such as for example, 25 m, 50 m, 100 m, 200 m, 500 m, or 1 km. As aspecific example, a lidar system with a 200-m maximum range may beconfigured to sense or identify various targets located up to 200 maway. For a lidar system with a 200-m maximum range (R_(MAX)=200 m), thetime of flight corresponding to the maximum range is approximately2·R_(MAX)/c≅1.33 μs.

In some implementations, the light source 110, the scanner 120, and thereceiver 140 may be packaged together within a single housing 155, whichmay be a box, case, or enclosure that holds or contains all or part of alidar system 100. The housing 155 includes a window 157 through whichthe beams 125 and 135 pass. In one example implementation, thelidar-system housing 155 contains the light source 110, the overlapmirror 115, the scanner 120, and the receiver 140 of a lidar system 100.The controller 150 may reside within the same housing 155 as thecomponents 110, 120, and 140, or the controller 150 may reside remotelyfrom the housing.

Moreover, in some implementations, the housing 155 includes multiplelidar sensors, each including a respective scanner and a receiver.Depending on the particular implementation, each of the multiple sensorscan include a separate light source or a common light source. Themultiple sensors can be configured to cover non-overlapping adjacentfields of regard or partially overlapping fields of regard, depending onthe implementation.

The housing 155 may be an airtight or watertight structure that preventswater vapor, liquid water, dirt, dust, or other contaminants fromgetting inside the housing 155. The housing 155 may be filled with a dryor inert gas, such as for example dry air, nitrogen, or argon. Thehousing 155 may include one or more electrical connections for conveyingelectrical power or electrical signals to and/or from the housing.

The window 157 may be made from any suitable substrate material, such asfor example, glass or plastic (e.g., polycarbonate, acrylic,cyclic-olefin polymer, or cyclic-olefin copolymer). The window 157 mayinclude an interior surface (surface A) and an exterior surface (surfaceB), and surface A or surface B may include a dielectric coating havingparticular reflectivity values at particular wavelengths. A dielectriccoating (which may be referred to as a thin-film coating, interferencecoating, or coating) may include one or more thin-film layers ofdielectric materials (e.g., SiO₂, TiO₂, Al₂O₃, Ta₂O₅, MgF₂, LaF₃, orAlF₃) having particular thicknesses (e.g., thickness less than 1 μm) andparticular refractive indices. A dielectric coating may be depositedonto surface A or surface B of the window 157 using any suitabledeposition technique, such as for example, sputtering or electron-beamdeposition.

The dielectric coating may have a high reflectivity at a particularwavelength or a low reflectivity at a particular wavelength. Ahigh-reflectivity (HR) dielectric coating may have any suitablereflectivity value (e.g., a reflectivity greater than or equal to 80%,90%, 95%, or 99%) at any suitable wavelength or combination ofwavelengths. A low-reflectivity dielectric coating (which may bereferred to as an anti-reflection (AR) coating) may have any suitablereflectivity value (e.g., a reflectivity less than or equal to 5%, 2%,1%, 0.5%, or 0.2%) at any suitable wavelength or combination ofwavelengths. In particular embodiments, a dielectric coating may be adichroic coating with a particular combination of high or lowreflectivity values at particular wavelengths. For example, a dichroiccoating may have a reflectivity of less than or equal to 0.5% atapproximately 1550-1560 nm and a reflectivity of greater than or equalto 90% at approximately 800-1500 nm.

In some implementations, surface A or surface B has a dielectric coatingthat is anti-reflecting at an operating wavelength of one or more lightsources 110 contained within enclosure 155. An AR coating on surface Aand surface B may increase the amount of light at an operatingwavelength of light source 110 that is transmitted through the window157. Additionally, an AR coating at an operating wavelength of the lightsource 110 may reduce the amount of incident light from output beam 125that is reflected by the window 157 back into the housing 155. In anexample implementation, each of surface A and surface B has an ARcoating with reflectivity less than 0.5% at an operating wavelength oflight source 110. As an example, if the light source 110 has anoperating wavelength of approximately 1550 nm, then surface A andsurface B may each have an AR coating with a reflectivity that is lessthan 0.5% from approximately 1547 nm to approximately 1553 nm. Inanother implementation, each of surface A and surface B has an ARcoating with reflectivity less than 1% at the operating wavelengths ofthe light source 110. For example, if the housing 155 encloses twosensor heads with respective light sources, the first light source emitspulses at a wavelength of approximately 1535 nm and the second lightsource emits pulses at a wavelength of approximately 1540 nm, thensurface A and surface B may each have an AR coating with reflectivityless than 1% from approximately 1530 nm to approximately 1545 nm.

The window 157 may have an optical transmission that is greater than anysuitable value for one or more wavelengths of one or more light sources110 contained within the housing 155. As an example, the window 157 mayhave an optical transmission of greater than or equal to 70%, 80%, 90%,95%, or 99% at a wavelength of light source 110. In one exampleimplementation, the window 157 can transmit greater than or equal to 95%of light at an operating wavelength of the light source 110. In anotherimplementation, the window 157 transmits greater than or equal to 90% oflight at the operating wavelengths of the light sources enclosed withinthe housing 155.

Surface A or surface B may have a dichroic coating that isanti-reflecting at one or more operating wavelengths of one or morelight sources 110 and high-reflecting at wavelengths away from the oneor more operating wavelengths. For example, surface A may have an ARcoating for an operating wavelength of the light source 110, and surfaceB may have a dichroic coating that is AR at the light-source operatingwavelength and HR for wavelengths away from the operating wavelength. Acoating that is HR for wavelengths away from a light-source operatingwavelength may prevent most incoming light at unwanted wavelengths frombeing transmitted through the window 117. In one implementation, iflight source 110 emits optical pulses with a wavelength of approximately1550 nm, then surface A may have an AR coating with a reflectivity ofless than or equal to 0.5% from approximately 1546 nm to approximately1554 nm. Additionally, surface B may have a dichroic coating that is ARat approximately 1546-1554 nm and HR (e.g., reflectivity of greater thanor equal to 90%) at approximately 800-1500 nm and approximately1580-1700 nm.

Surface B of the window 157 may include a coating that is oleophobic,hydrophobic, or hydrophilic. A coating that is oleophobic (or,lipophobic) may repel oils (e.g., fingerprint oil or other non-polarmaterial) from the exterior surface (surface B) of the window 157. Acoating that is hydrophobic may repel water from the exterior surface.For example, surface B may be coated with a material that is botholeophobic and hydrophobic. A coating that is hydrophilic attracts waterso that water may tend to wet and form a film on the hydrophilic surface(rather than forming beads of water as may occur on a hydrophobicsurface). If surface B has a hydrophilic coating, then water (e.g., fromrain) that lands on surface B may form a film on the surface. Thesurface film of water may result in less distortion, deflection, orocclusion of an output beam 125 than a surface with a non-hydrophiliccoating or a hydrophobic coating.

In some implementations, the window 157 includes one or severalultrasonic transducers 159 a, 159 b physically attached to the interiorsurface (surface A) of the window 157. For example, the ultrasonictransducers 159 a, 159 b are bonded to the window 157 or a window frame.The ultrasonic transducers 159 a, 159 b are then configured to vibrateat an ultrasonic frequency (e.g., 20 kHz, 50 kHz, 100 kHz, 1 MHz, 10MHz, etc.) to remove debris, such as water, fog, dirt, etc. from theexterior surface (surface B) of the window 157. In some implementations,a single ultrasonic transducer 159 a is used to generate a standingultrasonic wave and the vibration causes the debris to detach from thewindow 157. In other implementations, two ultrasonic transducers 159 a,159 b vibrate with a particular phase shift to generate a travellingultrasonic wave. The travelling ultrasonic wave causes the debris tosweep across surface B of the window 157.

The oleophobic and/or hydrophobic coating on surface B may also be usedin combination with the ultrasonic transducers 159 a, 159 b to removedebris. Additionally, windshield wipers and/or a nozzle that sprayswater or cleaner may be attached to surface B of the window 157. Whendebris is detected at the window 157, any suitable combination of thewindshield wipers, spray, hydrophobic or oleophobic layer, and vibrationmay be used to remove the debris.

In some implementations, the controller 150 is communicatively coupledto the ultrasonic transducers 159 a, 159 b, windshield wipers, and/ornozzle. When the controller 150 detects debris on surface B of thewindow or receives an indication of a likely presence of debris (e.g.,from the rain or weather sensor), the controller 150 provides controlsignals activating the ultrasonic transducers 159 a, 159 b, windshieldwipers, and/or nozzle. In some scenarios, the controller 150 adjusts theintensity of the vibration including the frequency and/or amplitude ofthe ultrasonic waves according to the amount, size, or type of debris onthe window. For example, the frequency may be 20 kHz when there are afew droplets of rain on the window. When the entire window is coveredwith debris, the frequency may be set to 10 MHz.

Debris may be detected via a rain or other weather sensor within thelidar system 100. In some implementations, the rain sensor detects rainby transmitting optical pulses at the window 157. When the window iswet, at least some of the optical pulses are not scattered and thenreceived by the rain sensor. Debris may also be detected based on thecharacteristics of return light pulses at the receiver, or when thereceiver does not detect return light pulses from the same receiverfield of view over a threshold period of time or number of frames. Thecontroller 150 may analyze the characteristics of the return lightpulses or lack thereof to detect debris. While the ultrasonictransducers 159 a, 159 b are illustrated in FIG. 1 as being attached tothe window 157, this is merely one exemplary implementation. In otherimplementations, the ultrasonic transducers 159 a, 159 b may be attachedto the window 212 as described in more detail below with reference toFIG. 3.

In some implementations, the window 157 includes two windows such as aninternal window for sealing the lidar system 100 and an external windowhaving the one or several ultrasonic transducers 159 a, 159 b physicallyattached. The external window may be a cover that protects the lidarsystem 100 but may not be airtight or watertight. The internal windowmay provide an airtight, watertight, or hermetic seal that preventswater vapor, liquid water, dirt, dust, or other contaminants fromgetting inside the lidar-system enclosure. Prior to being sealed, thelidar-system enclosure may be filled or purged with a dry or inert gas,such as for example, dry air, nitrogen, or argon.

With continued reference to FIG. 1, the light source 110 may include apulsed laser configured to produce or emit pulses of light with acertain pulse duration. In an example implementation, the pulse durationor pulse width of the pulsed laser is approximately 10 picoseconds (ps)to 20 nanoseconds (ns). In another implementation, the light source 110is a pulsed laser that produces pulses with a pulse duration ofapproximately 1-4 ns. In yet another implementation, the light source110 is a pulsed laser that produces pulses at a pulse repetitionfrequency of approximately 100 kHz to 5 MHz or a pulse period (e.g., atime between consecutive pulses) of approximately 200 ns to 10 μs. Thelight source 110 may have a substantially constant or a variable pulserepetition frequency, depending on the implementation. As an example,the light source 110 may be a pulsed laser that produces pulses at asubstantially constant pulse repetition frequency of approximately 640kHz (e.g., 640,000 pulses per second), corresponding to a pulse periodof approximately 1.56 μs. As another example, the light source 110 mayhave a pulse repetition frequency that can be varied from approximately500 kHz to 3 MHz. As used herein, a pulse of light may be referred to asan optical pulse, a light pulse, or a pulse, and a pulse repetitionfrequency may be referred to as a pulse rate.

In general, the output beam 125 may have any suitable average opticalpower, and the output beam 125 may include optical pulses with anysuitable pulse energy or peak optical power. Some examples of theaverage power of the output beam 125 include the approximate values of 1mW, 10 mW, 100 mW, 1 W, and 10 W. Example values of pulse energy of theoutput beam 125 include the approximate values of 0.1 μJ, 1 μJ, 10 μJ,100 μJ, and 1 mJ. Examples of peak power values of pulses included inthe output beam 125 are the approximate values of 10 W, 100 W, 1 kW, 5kW, 10 kW. An example optical pulse with a duration of 1 ns and a pulseenergy of 1 μJ has a peak power of approximately 1 kW. If the pulserepetition frequency is 500 kHz, then the average power of the outputbeam 125 with 1-μJ pulses is approximately 0.5 W, in this example.

The light source 110 may include a laser diode, such as a Fabry-Perotlaser diode, a quantum well laser, a distributed Bragg reflector (DBR)laser, a distributed feedback (DFB) laser, or a vertical-cavitysurface-emitting laser (VCSEL). The laser diode operating in the lightsource 110 may be an aluminum-gallium-arsenide (AlGaAs) laser diode, anindium-gallium-arsenide (InGaAs) laser diode, or anindium-gallium-arsenide-phosphide (InGaAsP) laser diode, or any othersuitable diode. In some implementations, the light source 110 includes apulsed laser diode with a peak emission wavelength of approximately1400-1600 nm. Further, the light source 110 may include a laser diodethat is current-modulated to produce optical pulses.

In some implementation, the light source 110 includes a pulsed laserdiode followed by one or more optical-amplification stages. For example,the light source 110 may be a fiber-laser module that includes acurrent-modulated laser diode with a peak wavelength of approximately1550 nm, followed by a single-stage or a multi-stage erbium-doped fiberamplifier (EDFA). As another example, the light source 110 may include acontinuous-wave (CW) or quasi-CW laser diode followed by an externaloptical modulator (e.g., an electro-optic modulator), and the output ofthe modulator may be fed into an optical amplifier. In yet otherimplementations, the light source 110 may include a pulsed solid-statelaser or a pulsed fiber laser.

In some implementations, the output beam of light 125 emitted by thelight source 110 is a collimated optical beam with any suitable beamdivergence, such as a divergence of approximately 0.1 to 3.0 milliradian(mrad). Divergence of the output beam 125 may refer to an angularmeasure of an increase in beam size (e.g., a beam radius or beamdiameter) as the output beam 125 travels away from the light source 110or the lidar system 100. The output beam 125 may have a substantiallycircular cross section with a beam divergence characterized by a singledivergence value. For example, the output beam 125 with a circular crosssection and a divergence of 1 mrad may have a beam diameter or spot sizeof approximately 10 cm at a distance of 100 m from the lidar system 100.In some implementations, the output beam 125 may be an astigmatic beamor may have a substantially elliptical cross section and may becharacterized by two divergence values. As an example, the output beam125 may have a fast axis and a slow axis, where the fast-axis divergenceis greater than the slow-axis divergence. As another example, the outputbeam 125 may be an astigmatic beam with a fast-axis divergence of 2 mradand a slow-axis divergence of 0.5 mrad.

The output beam of light 125 emitted by light source 110 may beunpolarized or randomly polarized, may have no specific or fixedpolarization (e.g., the polarization may vary with time), or may have aparticular polarization (e.g., the output beam 125 may be linearlypolarized, elliptically polarized, or circularly polarized). As anexample, the light source 110 may produce linearly polarized light, andthe lidar system 100 may include a quarter-wave plate that converts thislinearly polarized light into circularly polarized light. The lidarsystem 100 may transmit the circularly polarized light as the outputbeam 125, and receive the input beam 135, which may be substantially orat least partially circularly polarized in the same manner as the outputbeam 125 (e.g., if the output beam 125 is right-hand circularlypolarized, then the input beam 135 may also be right-hand circularlypolarized). The input beam 135 may pass through the same quarter-waveplate (or a different quarter-wave plate), resulting in the input beam135 being converted to linearly polarized light which is orthogonallypolarized (e.g., polarized at a right angle) with respect to thelinearly polarized light produced by light source 110. As anotherexample, the lidar system 100 may employ polarization-diversitydetection where two polarization components are detected separately. Theoutput beam 125 may be linearly polarized, and the lidar system 100 maysplit the input beam 135 into two polarization components (e.g.,s-polarization and p-polarization) which are detected separately by twophotodiodes (e.g., a balanced photoreceiver that includes twophotodiodes).

With continued reference to FIG. 1, the output beam 125 and input beam135 may be substantially coaxial. In other words, the output beam 125and input beam 135 may at least partially overlap or share a commonpropagation axis, so that the input beam 135 and the output beam 125travel along substantially the same optical path (albeit in oppositedirections). As the lidar system 100 scans the output beam 125 across afield of regard, the input beam 135 may follow along with the outputbeam 125, so that the coaxial relationship between the two beams ismaintained.

The lidar system 100 also may include one or more optical componentsconfigured to condition, shape, filter, modify, steer, or direct theoutput beam 125 and/or the input beam 135. For example, lidar system 100may include one or more lenses, mirrors, filters (e.g., bandpass orinterference filters), beam splitters, polarizers, polarizing beamsplitters, wave plates (e.g., half-wave or quarter-wave plates),diffractive elements, or holographic elements. In some implementations,lidar system 100 includes a telescope, one or more lenses, or one ormore mirrors to expand, focus, or collimate the output beam 125 to adesired beam diameter or divergence. As an example, the lidar system 100may include one or more lenses to focus the input beam 135 onto anactive region of the receiver 140. As another example, the lidar system100 may include one or more flat mirrors or curved mirrors (e.g.,concave, convex, or parabolic mirrors) to steer or focus the output beam125 or the input beam 135. For example, the lidar system 100 may includean off-axis parabolic mirror to focus the input beam 135 onto an activeregion of receiver 140. As illustrated in FIG. 1, the lidar system 100may include the mirror 115, which may be a metallic or dielectricmirror. The mirror 115 may be configured so that the light beam 125passes through the mirror 115. As an example, mirror 115 may include ahole, slot, or aperture through which the output light beam 125 passes.As another example, the mirror 115 may be configured so that at least80% of the output beam 125 passes through the mirror 115 and at least80% of the input beam 135 is reflected by the mirror 115. In someimplementations, the mirror 115 may provide for the output beam 125 andthe input beam 135 to be substantially coaxial, so that the beams 125and 135 travel along substantially the same optical path, in oppositedirections.

Generally speaking, the scanner 120 steers the output beam 125 in one ormore directions downrange. The scanner 120 may include one or morescanning mirrors and one or more actuators driving the mirrors torotate, tilt, pivot, or move the mirrors in an angular manner about oneor more axes, for example. For example, the first mirror of the scannermay scan the output beam 125 along a first direction, and the secondmirror may scan the output beam 125 along a second direction that issubstantially orthogonal to the first direction. Example implementationsof the scanner 120 are discussed in more detail below with reference toFIG. 2.

The scanner 120 may be configured to scan the output beam 125 over a5-degree angular range, 20-degree angular range, 30-degree angularrange, 60-degree angular range, or any other suitable angular range. Forexample, a scanning mirror may be configured to periodically rotate overa 15-degree range, which results in the output beam 125 scanning acrossa 30-degree range (e.g., a 0-degree rotation by a scanning mirrorresults in a 20-degree angular scan of the output beam 125). A field ofregard (FOR) of the lidar system 100 may refer to an area, region, orangular range over which the lidar system 100 may be configured to scanor capture distance information. When the lidar system 100 scans theoutput beam 125 within a 30-degree scanning range, the lidar system 100may be referred to as having a 30-degree angular field of regard. Asanother example, a lidar system 100 with a scanning mirror that rotatesover a 30-degree range may produce the output beam 125 that scans acrossa 60-degree range (e.g., a 60-degree FOR). In various implementations,the lidar system 100 may have a FOR of approximately 10°, 20°, 40°, 60°,120°, or any other suitable FOR. The FOR also may be referred to as ascan region.

The scanner 120 may be configured to scan the output beam 125horizontally and vertically, and the lidar system 100 may have aparticular FOR along the horizontal direction and another particular FORalong the vertical direction. For example, the lidar system 100 may havea horizontal FOR of 10° to 120° and a vertical FOR of 2° to 45°.

The one or more scanning mirrors of the scanner 120 may becommunicatively coupled to the controller 150 which may control thescanning mirror(s) so as to guide the output beam 125 in a desireddirection downrange or along a desired scan pattern. In general, a scanpattern may refer to a pattern or path along which the output beam 125is directed, and also may be referred to as an optical scan pattern,optical scan path, or scan path. As an example, the scanner 120 mayinclude two scanning mirrors configured to scan the output beam 125across a 60° horizontal FOR and a 20° vertical FOR. The two scannermirrors may be controlled to follow a scan path that substantiallycovers the 60°×20° FOR. The lidar system 100 can use the scan path togenerate a point cloud with pixels that substantially cover the 60°×20°FOR. The pixels may be approximately evenly distributed across the60°×20° FOR. Alternately, the pixels may have a particular non-uniformdistribution (e.g., the pixels may be distributed across all or aportion of the 60°×20° FOR, and the pixels may have a higher density inone or more particular regions of the 60°×20° FOR).

In operation, the light source 110 may emit pulses of light which thescanner 120 scans across a FOR of lidar system 100. The target 130 mayscatter one or more of the emitted pulses, and the receiver 140 maydetect at least a portion of the pulses of light scattered by the target130.

The receiver 140 may be referred to as (or may include) a photoreceiver,optical receiver, optical sensor, detector, photodetector, or opticaldetector. The receiver 140 in some implementations receives or detectsat least a portion of the input beam 135 and produces an electricalsignal that corresponds to the input beam 135. For example, if the inputbeam 135 includes an optical pulse, then the receiver 140 may produce anelectrical current or voltage pulse that corresponds to the opticalpulse detected by the receiver 140. In an example implementation, thereceiver 140 includes one or more avalanche photodiodes (APDs) or one ormore single-photon avalanche diodes (SPADs). In another implementation,the receiver 140 includes one or more PN photodiodes (e.g., a photodiodestructure formed by a p-type semiconductor and a n-type semiconductor)or one or more PIN photodiodes (e.g., a photodiode structure formed byan undoped intrinsic semiconductor region located between p-type andn-type regions).

The receiver 140 may have an active region or anavalanche-multiplication region that includes silicon, germanium, orInGaAs. The active region of receiver 140 may have any suitable size,such as for example, a diameter or width of approximately 50-500 μm. Thereceiver 140 may include circuitry that performs signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection. For example, thereceiver 140 may include a transimpedance amplifier that converts areceived photocurrent (e.g., a current produced by an APD in response toa received optical signal) into a voltage signal. The receiver 140 maydirect the voltage signal to pulse-detection circuitry that produces ananalog or digital output signal 145 that corresponds to one or morecharacteristics (e.g., rising edge, falling edge, amplitude, orduration) of a received optical pulse. For example, the pulse-detectioncircuitry may perform a time-to-digital conversion to produce a digitaloutput signal 145. The receiver 140 may send the electrical outputsignal 145 to the controller 150 for processing or analysis, e.g., todetermine a time-of-flight value corresponding to a received opticalpulse.

The controller 150 may be electrically coupled or otherwisecommunicatively coupled to one or more of the light source 110, thescanner 120, and the receiver 140. The controller 150 may receiveelectrical trigger pulses or edges from the light source 110, where eachpulse or edge corresponds to the emission of an optical pulse by thelight source 110. The controller 150 may provide instructions, a controlsignal, or a trigger signal to the light source 110 indicating when thelight source 110 should produce optical pulses. For example, thecontroller 150 may send an electrical trigger signal that includeselectrical pulses, where the light source 110 emits an optical pulse inresponse to each electrical pulse. Further, the controller 150 may causethe light source 110 to adjust one or more of the frequency, period,duration, pulse energy, peak power, average power, or wavelength of theoptical pulses produced by the light source 110.

The controller 150 may determine a time-of-flight value for an opticalpulse based on timing information associated with when the pulse wasemitted by light source 110 and when a portion of the pulse (e.g., theinput beam 135) was detected or received by the receiver 140. Thecontroller 150 may include circuitry that performs signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection.

As indicated above, the lidar system 100 may be used to determine thedistance to one or more downrange targets 130. By scanning the lidarsystem 100 across a field of regard, the system can be used to map thedistance to a number of points within the field of regard. Each of thesedepth-mapped points may be referred to as a pixel or a voxel. Acollection of pixels captured in succession (which may be referred to asa depth map, a point cloud, or a frame) may be rendered as an image ormay be analyzed to identify or detect objects or to determine a shape ordistance of objects within the FOR. For example, a depth map may cover afield of regard that extends 60° horizontally and 15° vertically, andthe depth map may include a frame of 100-2000 pixels in the horizontaldirection by 4-400 pixels in the vertical direction.

The lidar system 100 may be configured to repeatedly capture or generatepoint clouds of a field of regard at any suitable frame rate betweenapproximately 0.1 frames per second (FPS) and approximately 1,000 FPS.For example, the lidar system 100 may generate point clouds at a framerate of approximately 0.1 FPS, 0.5 FPS, 1 FPS, 2 FPS, 5 FPS, 10 FPS, 20FPS, 100 FPS, 500 FPS, or 1,000 FPS. In an example implementation, thelidar system 100 is configured to produce optical pulses at a rate of5×10⁵ pulses/second (e.g., the system may determine 500,000 pixeldistances per second) and scan a frame of 1000×50 pixels (e.g., 50,000pixels/frame), which corresponds to a point-cloud frame rate of 10frames per second (e.g., 10 point clouds per second). The point-cloudframe rate may be substantially fixed or dynamically adjustable,depending on the implementation. For example, the lidar system 100 maycapture one or more point clouds at a particular frame rate (e.g., 1 Hz)and then switch to capture one or more point clouds at a different framerate (e.g., 10 Hz). In general, the lidar system can use a slower framerate (e.g., 1 Hz) to capture one or more high-resolution point clouds,and use a faster frame rate (e.g., 10 Hz) to rapidly capture multiplelower-resolution point clouds.

The field of regard of the lidar system 100 can overlap, encompass, orenclose at least a portion of the target 130, which may include all orpart of an object that is moving or stationary relative to lidar system100. For example, the target 130 may include all or a portion of aperson, vehicle, motorcycle, truck, train, bicycle, wheelchair,pedestrian, animal, road sign, traffic light, lane marking, road-surfacemarking, parking space, pylon, guard rail, traffic barrier, pothole,railroad crossing, obstacle in or near a road, curb, stopped vehicle onor beside a road, utility pole, house, building, trash can, mailbox,tree, any other suitable object, or any suitable combination of all orpart of two or more objects.

Now referring to FIG. 2, a scanner 162 and a receiver 164 can operate inthe lidar system of FIG. 1 as the scanner 120 and the receiver 140,respectively. More generally, the scanner 162 and the receiver 164 canoperate in any suitable lidar system.

The scanner 162 may include any suitable number of mirrors driven by anysuitable number of mechanical actuators. For example, the scanner 162may include a galvanometer scanner, a resonant scanner, a piezoelectricactuator, a polygonal scanner, a rotating-prism scanner, a voice coilmotor, a DC motor, a brushless DC motor, a stepper motor, or amicroelectromechanical systems (MEMS) device, or any other suitableactuator or mechanism.

A galvanometer scanner (which also may be referred to as a galvanometeractuator) may include a galvanometer-based scanning motor with a magnetand coil. When an electrical current is supplied to the coil, arotational force is applied to the magnet, which causes a mirrorattached to the galvanometer scanner to rotate. The electrical currentsupplied to the coil may be controlled to dynamically change theposition of the galvanometer mirror. A resonant scanner (which may bereferred to as a resonant actuator) may include a spring-like mechanismdriven by an actuator to produce a periodic oscillation at asubstantially fixed frequency (e.g., 1 kHz). A MEMS-based scanningdevice may include a mirror with a diameter between approximately 1 and10 mm, where the mirror is rotated using electromagnetic orelectrostatic actuation. A voice coil motor (which may be referred to asa voice coil actuator) may include a magnet and coil. When an electricalcurrent is supplied to the coil, a translational force is applied to themagnet, which causes a mirror attached to the magnet to move or rotate.

In an example implementation, the scanner 162 includes a single mirrorconfigured to scan an output beam 170 along a single direction (e.g.,the scanner 162 may be a one-dimensional scanner that scans along ahorizontal or vertical direction). The mirror may be a flat scanningmirror attached to a scanner actuator or mechanism which scans themirror over a particular angular range. The mirror may be driven by oneactuator (e.g., a galvanometer) or two actuators configured to drive themirror in a push-pull configuration. When two actuators drive the mirrorin one direction in a push-pull configuration, the actuators may belocated at opposite ends or sides of the mirror. The actuators mayoperate in a cooperative manner so that when one actuator pushes on themirror, the other actuator pulls on the mirror, and vice versa. Inanother example implementation, two voice coil actuators arranged in apush-pull configuration drive a mirror along a horizontal or verticaldirection.

In some implementations, the scanner 162 may include one mirrorconfigured to be scanned along two axes, where two actuators arranged ina push-pull configuration provide motion along each axis. For example,two resonant actuators arranged in a horizontal push-pull configurationmay drive the mirror along a horizontal direction, and another pair ofresonant actuators arranged in a vertical push-pull configuration maydrive mirror along a vertical direction. In another exampleimplementation, two actuators scan the output beam 170 along twodirections (e.g., horizontal and vertical), where each actuator providesrotational motion along a particular direction or about a particularaxis.

The scanner 162 also may include one mirror driven by two actuatorsconfigured to scan the mirror along two substantially orthogonaldirections. For example, a resonant actuator or a galvanometer actuatormay drive one mirror along a substantially horizontal direction, and agalvanometer actuator may drive the mirror along a substantiallyvertical direction. As another example, two resonant actuators may drivea mirror along two substantially orthogonal directions.

In some implementations, the scanner 162 includes two mirrors, where onemirror scans the output beam 170 along a substantially horizontaldirection and the other mirror scans the output beam 170 along asubstantially vertical direction. In the example of FIG. 2, the scanner162 includes two mirrors, a mirror 180-1 and a mirror 180-2. The mirror180-1 may scan the output beam 170 along a substantially horizontaldirection, and the mirror 180-2 may scan the output beam 170 along asubstantially vertical direction (or vice versa). Mirror 180-1 or mirror180-2 may be a flat mirror, a curved mirror, or a polygon mirror withtwo or more reflective surfaces.

The scanner 162 in other implementations includes two galvanometerscanners driving respective mirrors. For example, the scanner 162 mayinclude a galvanometer actuator that scans the mirror 180-1 along afirst direction (e.g., vertical), and the scanner 162 may includeanother galvanometer actuator that scans the mirror 180-2 along a seconddirection (e.g., horizontal). In yet another implementation, the scanner162 includes two mirrors, where a galvanometer actuator drives onemirror, and a resonant actuator drives the other mirror. For example, agalvanometer actuator may scan the mirror 180-1 along a first direction,and a resonant actuator may scan the mirror 180-2 along a seconddirection. The first and second scanning directions may be substantiallyorthogonal to one another, e.g., the first direction may besubstantially vertical, and the second direction may be substantiallyhorizontal. In yet another implementation, the scanner 162 includes twomirrors, where one mirror is a polygon mirror that is rotated in onedirection (e.g., clockwise or counter-clockwise) by an electric motor(e.g., a brushless DC motor). For example, mirror 180-1 may be a polygonmirror that scans the output beam 170 along a substantially horizontaldirection, and mirror 180-2 may scan the output beam 170 along asubstantially vertical direction. A polygon mirror may have two or morereflective surfaces, and the polygon mirror may be continuously rotatedin one direction so that the output beam 170 is reflected sequentiallyfrom each of the reflective surfaces. A polygon mirror may have across-sectional shape that corresponds to a polygon, where each side ofthe polygon has a reflective surface. For example, a polygon mirror witha square cross-sectional shape may have four reflective surfaces, and apolygon mirror with a pentagonal cross-sectional shape may have fivereflective surfaces.

To direct the output beam 170 along a particular scan pattern, thescanner 162 may include two or more actuators driving a single mirrorsynchronously. For example, the two or more actuators can drive themirror synchronously along two substantially orthogonal directions tomake the output beam 170 follow a scan pattern with substantiallystraight lines. In some implementations, the scanner 162 may include twomirrors and actuators driving the two mirrors synchronously to generatea scan pattern that includes substantially straight lines. For example,a galvanometer actuator may drive the mirror 180-2 with a substantiallylinear back-and-forth motion (e.g., the galvanometer may be driven witha substantially sinusoidal or triangle-shaped waveform) that causes theoutput beam 170 to trace a substantially horizontal back-and-forthpattern, and another galvanometer actuator may scan the mirror 180-1along a substantially vertical direction. The two galvanometers may besynchronized so that for every 64 horizontal traces, the output beam 170makes a single trace along a vertical direction. Whether one or twomirrors are used, the substantially straight lines can be directedsubstantially horizontally, vertically, or along any other suitabledirection.

The scanner 162 also may apply a dynamically adjusted deflection along avertical direction (e.g., with a galvanometer actuator) as the outputbeam 170 is scanned along a substantially horizontal direction (e.g.,with a galvanometer or resonant actuator) to achieve the straight lines.If a vertical deflection is not applied, the output beam 170 may traceout a curved path as it scans from side to side. In someimplementations, the scanner 162 uses a vertical actuator to apply adynamically adjusted vertical deflection as the output beam 170 isscanned horizontally as well as a discrete vertical offset between eachhorizontal scan (e.g., to step the output beam 170 to a subsequent rowof a scan pattern).

With continued reference to FIG. 2, an overlap mirror 190 in thisexample implementation is configured to overlap the input beam 172 andoutput beam 170, so that the beams 170 and 172 are substantiallycoaxial. In FIG. 2, the overlap mirror 190 includes a hole, slot, oraperture 192 through which the output beam 170 passes, and a reflectingsurface 194 that reflects at least a portion of the input beam 172toward the receiver 164. The overlap mirror 190 may be oriented so thatinput beam 172 and output beam 170 are at least partially overlapped.

In some implementations, the overlap mirror 190 may not include a hole192. For example, the output beam 170 may be directed to pass by a sideof mirror 190 rather than passing through an aperture 192. The outputbeam 170 may pass alongside mirror 190 and may be oriented at a slightangle with respect to the orientation of the input beam 172. As anotherexample, the overlap mirror 190 may include a small reflective sectionconfigured to reflect the output beam 170, and the rest of the overlapmirror 190 may have an AR coating configured to transmit the input beam172.

The input beam 172 may pass through a lens 196 which focuses the beamonto an active region 166 of the receiver 164. The active region 166 mayrefer to an area over which receiver 164 may receive or detect inputlight. The active region may have any suitable size or diameter d, suchas for example, a diameter of approximately 25 μm, 50 μm, 80 μm, 100 μm,200 μm, 500 μm, 1 mm, 2 mm, or 5 mm. The overlap mirror 190 may have areflecting surface 194 that is substantially flat or the reflectingsurface 194 may be curved (e.g., the mirror 190 may be an off-axisparabolic mirror configured to focus the input beam 172 onto an activeregion of the receiver 140).

The aperture 192 may have any suitable size or diameter Φ₁, and theinput beam 172 may have any suitable size or diameter Φ₂, where Φ₂ isgreater than Φ₁. For example, the aperture 192 may have a diameter Φ₁ ofapproximately 0.2 mm, 0.5 mm, 1 mm, 2 mm, 3 mm, 5 mm, or 10 mm, and theinput beam 172 may have a diameter Φ₂ of approximately 2 mm, 5 mm, 10mm, 15 mm, 20 mm, 30 mm, 40 mm, or 50 mm. In some implementations, thereflective surface 194 of the overlap mirror 190 may reflect 70% or moreof input beam 172 toward the receiver 164. For example, if thereflective surface 194 has a reflectivity R at an operating wavelengthof the light source 160, then the fraction of input beam 172 directedtoward the receiver 164 may be expressed as R×[1−(Φ₁/Φ₂)²]. As a morespecific example, if R is 95%, Φ₁ is 2 mm, and Φ₂ is 10 mm, thenapproximately 91% of the input beam 172 may be directed toward thereceiver 164 by the reflective surface 194.

FIG. 3 illustrates an example configuration in which several componentsof the lidar system 100 may operate to scan a 360-degree view of regard.Generally speaking, the field of view of a light source in thisconfiguration follows a circular trajectory and accordingly defines acircular scan pattern on a two-dimensional plane. All points on thetrajectory remain at the same elevation relative to the ground level,according to one implementation. In this case, separate beams may followthe circular trajectory with certain vertical offsets relative to eachother. In another implementation, the points of the trajectory maydefine a spiral scan pattern in three-dimensional space. A single beamcan be sufficient to trace out the spiral scan pattern but, if desired,multiple beams can be used.

In the example of FIG. 3, a rotating scan module 200 revolves around acentral axis in one or both directions as indicated. An electric motormay drive the rotating scan module 200 around the central axis at aconstant speed, for example. The rotating scan module 200 includes ascanner, a receiver, an overlap mirror, etc. The components of therotating module 200 may be similar to the scanner 120, the receiver 140,and the overlap mirror 115. In some implementations, the subsystem 200also includes a light source and a controller. In other implementations,the light source and/or the controller are disposed apart from therotating scan module 200 and/or exchange optical and electrical signalswith the components of the rotating scan module 200 via correspondinglinks.

The rotating scan module 200 may include a housing 210 with a window212. Similar to the window 157 of FIG. 1, the window 212 may be made ofglass, plastic, or any other suitable material. The window 212 allowsoutbound beams as well as return signals to pass through the housing210. The arc length defined by the window 212 can correspond to anysuitable percentage of the circumference of the housing 210. Forexample, the arc length can correspond to 5%, 20%, 30%, 60%, or possiblyeven 100% of the circumference.

Now referring to FIG. 4, a rotating scan module 220 is generally similarto the rotating scan module 200. In this implementation, however, thecomponents of the rotating scan module 220 are disposed on a platform222 which rotates inside a stationary circular housing 230. In thisimplementation, the circular housing 230 is substantially transparent tolight at the lidar-system operating wavelength to pass inbound andoutbound light signals. The circular housing 230 in a sense defines acircular window similar to the window 212, and may be made of similarmaterial.

Generating Pixels within a Field of Regard

FIG. 5 illustrates an example scan pattern 240 which the lidar system100 of FIG. 1 can produce. The lidar system 100 may be configured toscan output optical beam 125 along one or more scan patterns 240. Insome implementations, the scan pattern 240 corresponds to a scan acrossany suitable field of regard (FOR) having any suitable horizontal FOR(FOR_(H)) and any suitable vertical FOR (FOR_(V)). For example, acertain scan pattern may have a field of regard represented by angulardimensions (e.g., FOR_(H)×FOR_(V)) 40°×30°, 90°×40°, or 60°×15°. Asanother example, a certain scan pattern may have a FOR_(H) greater thanor equal to 10°, 25°, 30°, 40°, 60°, 90°, or 120°. As yet anotherexample, a certain scan pattern may have a FOR_(V) greater than or equalto 2°, 5°, 10°, 15°, 20°, 30°, or 45°. In the example of FIG. 5,reference line 246 represents a center of the field of regard of scanpattern 240. The reference line 246 may have any suitable orientation,such as, a horizontal angle of 0° (e.g., reference line 246 may beoriented straight ahead) and a vertical angle of 0° (e.g., referenceline 246 may have an inclination of 0°), or the reference line 246 mayhave a nonzero horizontal angle or a nonzero inclination (e.g., avertical angle of +10° or −10°). In FIG. 5, if the scan pattern 240 hasa 60°×15° field of regard, then the scan pattern 240 covers a ±30°horizontal range with respect to reference line 246 and a ±7.5° verticalrange with respect to reference line 246. Additionally, the optical beam125 in FIG. 5 has an orientation of approximately −15° horizontal and+3° vertical with respect to reference line 246. The beam 125 may bereferred to as having an azimuth of −15° and an altitude of +3° relativeto the reference line 246. An azimuth (which may be referred to as anazimuth angle) may represent a horizontal angle with respect to thereference line 246, and an altitude (which may be referred to as analtitude angle, elevation, or elevation angle) may represent a verticalangle with respect to the reference line 246.

The scan pattern 240 may include multiple pixels 242, and each pixel 242may be associated with one or more laser pulses and one or morecorresponding distance measurements. A cycle of scan pattern 240 mayinclude a total of P_(x)×P_(y) pixels 242 (e.g., a two-dimensionaldistribution of P_(x) by P_(y) pixels). For example, the scan pattern240 may include a distribution with dimensions of approximately100-2,000 pixels 242 along a horizontal direction and approximately4-400 pixels 242 along a vertical direction. As another example, thescan pattern 240 may include a distribution of 1,000 pixels 242 alongthe horizontal direction by 64 pixels 242 along the vertical direction(e.g., the frame size is 1000×64 pixels) for a total of 64,000 pixelsper cycle of scan pattern 240. The number of pixels 242 along ahorizontal direction may be referred to as a horizontal resolution ofthe scan pattern 240, and the number of pixels 242 along a verticaldirection may be referred to as a vertical resolution of the scanpattern 240. As an example, the scan pattern 240 may have a horizontalresolution of greater than or equal to 100 pixels 242 and a verticalresolution of greater than or equal to 4 pixels 242. As another example,the scan pattern 240 may have a horizontal resolution of 100-2,000pixels 242 and a vertical resolution of 4-400 pixels 242.

Each pixel 242 may be associated with a distance (e.g., a distance to aportion of a target 130 from which the corresponding laser pulse wasscattered) or one or more angular values. As an example, the pixel 242may be associated with a distance value and two angular values (e.g., anazimuth and altitude) that represent the angular location of the pixel242 with respect to the lidar system 100. A distance to a portion of thetarget 130 may be determined based at least in part on a time-of-flightmeasurement for a corresponding pulse. An angular value (e.g., anazimuth or altitude) may correspond to an angle (e.g., relative toreference line 246) of the output beam 125 (e.g., when a correspondingpulse is emitted from lidar system 100) or an angle of the input beam135 (e.g., when an input signal is received by lidar system 100). Insome implementations, the lidar system 100 determines an angular valuebased at least in part on a position of a component of the scanner 120.For example, an azimuth or altitude value associated with the pixel 242may be determined from an angular position of one or more correspondingscanning mirrors of the scanner 120.

In some implementations, the lidar system 100 concurrently directsmultiple beams across the field of regard. In the example implementationof FIG. 6, the lidar system generates output beams 250A, 250B, 250C, . .. 250N etc., each of which follows a linear scan pattern 254A, 254B,254C, . . . 254N. The number of parallel lines can be 2, 4, 12, 20, orany other suitable number. The lidar system 100 may angularly separatethe beams 250A, 250B, 250C, . . . 250N, so that, for example, theseparation between beams 250A and 250B at a certain distance may be 30cm, and the separation between the same beams 250A and 250B at a longerdistance may be 50 cm.

Similar to the scan pattern 240, each of the linear scan patterns 254A-Nincludes pixels associated with one or more laser pulses and distancemeasurements. FIG. 6 illustrates example pixels 252A, 252B and 252Calong the scan patterns 254A, 254B and 254C, respectively. The lidarsystem 100 in this example may generate the values for the pixels252A-252N at the same time, thus increasing the rate at which values forpixels are determined.

Depending on the implementation, the lidar system 100 may output thebeams 250A-N at the same wavelength or different wavelengths. The beam250A for example may have the wavelength of 1540 nm, the beam 250B mayhave the wavelength of 1550 nm, the beam 250C may have the wavelength of1560 nm, etc. The number of different wavelengths the lidar system 100uses need not match the number of beams. Thus, the lidar system 100 inthe example implementation of FIG. 6 may use M wavelengths with N beams,where 1≤M≤N.

Next, FIG. 7 illustrates an example light-source field of view (FOV_(L))and receiver field of view (FOV_(R)) for the lidar system 100. The lightsource 110 may emit pulses of light as the FOV_(L) and FOV_(R) arescanned by the scanner 120 across a field of regard (FOR). Thelight-source field of view may refer to an angular cone illuminated bythe light source 110 at a particular instant of time. Similarly, areceiver field of view may refer to an angular cone over which thereceiver 140 may receive or detect light at a particular instant oftime, and any light outside the receiver field of view may not bereceived or detected. For example, as the scanner 120 scans thelight-source field of view across a field of regard, the lidar system100 may send the pulse of light in the direction the FOV_(L) is pointingat the time the light source 110 emits the pulse. The pulse of light mayscatter off the target 130, and the receiver 140 may receive and detecta portion of the scattered light that is directed along or containedwithin the FOV_(R).

In some implementations, the scanner 120 is configured to scan both alight-source field of view and a receiver field of view across a fieldof regard of the lidar system 100. The lidar system 100 may emit anddetect multiple pulses of light as the scanner 120 scans the FOV_(L) andFOV_(R) across the field of regard while tracing out the scan pattern240. The scanner 120 in some implementations scans the light-sourcefield of view and the receiver field of view synchronously with respectto one another. In this case, as the scanner 120 scans FOV_(L) across ascan pattern 240, the FOV_(R) follows substantially the same path at thesame scanning speed. Additionally, the FOV_(L) and FOV_(R) may maintainthe same relative position to one another as the scanner 120 scansFOV_(L) and FOV_(R) across the field of regard. For example, the FOV_(L)may be substantially overlapped with or centered inside the FOV_(R) (asillustrated in FIG. 7), and the scanner 120 may maintain this relativepositioning between FOV_(L) and FOV_(R) throughout a scan. As anotherexample, the FOV_(R) may lag behind the FOV_(L) by a particular, fixedamount throughout a scan (e.g., the FOV_(R) may be offset from theFOV_(L) in a direction opposite the scan direction).

The FOV_(L) may have an angular size or extent Θ_(L) that issubstantially the same as or that corresponds to the divergence of theoutput beam 125, and the FOV_(R) may have an angular size or extentΘ_(R) that corresponds to an angle over which the receiver 140 mayreceive and detect light. The receiver field of view may be any suitablesize relative to the light-source field of view. For example, thereceiver field of view may be smaller than, substantially the same sizeas, or larger than the angular extent of the light-source field of view.In some implementations, the light-source field of view has an angularextent of less than or equal to 50 milliradians, and the receiver fieldof view has an angular extent of less than or equal to 50 milliradians.The FOV_(L) may have any suitable angular extent Θ_(L), such as forexample, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. Similarly,the FOV_(R) may have any suitable angular extent Θ_(R), such as forexample, approximately 0.1 mrad, 0.2 mrad, 0.5 mrad, 1 mrad, 1.5 mrad, 2mrad, 3 mrad, 5 mrad, 10 mrad, 20 mrad, 40 mrad, or 50 mrad. Thelight-source field of view and the receiver field of view may haveapproximately equal angular extents. As an example, Θ_(L) and Θ_(R) mayboth be approximately equal to 1 mrad, 2 mrad, or 3 mrad. In someimplementations, the receiver field of view is larger than thelight-source field of view, or the light-source field of view is largerthan the receiver field of view. For example, Θ_(L) may be approximatelyequal to 1.5 mrad, and Θ_(R) may be approximately equal to 3 mrad.

A pixel 242 may represent or correspond to a light-source field of view.As the output beam 125 propagates from the light source 110, thediameter of the output beam 125 (as well as the size of thecorresponding pixel 242) may increase according to the beam divergenceΘ_(L). As an example, if the output beam 125 has a Θ_(L) of 2 mrad, thenat a distance of 100 m from the lidar system 100, the output beam 125may have a size or diameter of approximately 20 cm, and a correspondingpixel 242 may also have a corresponding size or diameter ofapproximately 20 cm. At a distance of 200 m from the lidar system 100,the output beam 125 and the corresponding pixel 242 may each have adiameter of approximately 40 cm.

A Lidar System Operating in a Vehicle

As indicated above, one or more lidar systems 100 may be integrated intoa vehicle. In one example implementation, multiple lidar systems 100 maybe integrated into a car to provide a complete 360-degree horizontal FORaround the car. As another example, 4-10 lidar systems 100, each systemhaving a 45-degree to 90-degree horizontal FOR, may be combined togetherto form a sensing system that provides a point cloud covering a360-degree horizontal FOR. The lidar systems 100 may be oriented so thatadjacent FORs have an amount of spatial or angular overlap to allow datafrom the multiple lidar systems 100 to be combined or stitched togetherto form a single or continuous 360-degree point cloud. As an example,the FOR of each lidar system 100 may have approximately 1-15 degrees ofoverlap with an adjacent FOR. In particular embodiments, a vehicle mayrefer to a mobile machine configured to transport people or cargo. Forexample, a vehicle may include, may take the form of, or may be referredto as a car, automobile, motor vehicle, truck, bus, van, trailer,off-road vehicle, farm vehicle, lawn mower, construction equipment, golfcart, motorhome, taxi, motorcycle, scooter, bicycle, skateboard, train,snowmobile, watercraft (e.g., a ship or boat), aircraft (e.g., afixed-wing aircraft, helicopter, or dirigible), or spacecraft. Inparticular embodiments, a vehicle may include an internal combustionengine or an electric motor that provides propulsion for the vehicle.

In some implementations, one or more lidar systems 100 are included in avehicle as part of an advanced driver assistance system (ADAS) to assista driver of the vehicle in the driving process. For example, a lidarsystem 100 may be part of an ADAS that provides information or feedbackto a driver (e.g., to alert the driver to potential problems or hazards)or that automatically takes control of part of a vehicle (e.g., abraking system or a steering system) to avoid collisions or accidents.The lidar system 100 may be part of a vehicle ADAS that providesadaptive cruise control, automated braking, automated parking, collisionavoidance, alerts the driver to hazards or other vehicles, maintains thevehicle in the correct lane, or provides a warning if an object oranother vehicle is in a blind spot.

In some cases, one or more lidar systems 100 are integrated into avehicle as part of an autonomous-vehicle driving system. In an exampleimplementation, the lidar system 100 provides information about thesurrounding environment to a driving system of an autonomous vehicle. Anautonomous-vehicle driving system may include one or more computingsystems that receive information from the lidar system 100 about thesurrounding environment, analyze the received information, and providecontrol signals to the vehicle's driving systems (e.g., steering wheel,accelerator, brake, or turn signal). For example, the lidar system 100integrated into an autonomous vehicle may provide an autonomous-vehicledriving system with a point cloud every 0.1 seconds (e.g., the pointcloud has a 10 Hz update rate, representing 10 frames per second). Theautonomous-vehicle driving system may analyze the received point cloudsto sense or identify targets 130 and their respective locations,distances, or speeds, and the autonomous-vehicle driving system mayupdate control signals based on this information. As an example, if thelidar system 100 detects a vehicle ahead that is slowing down orstopping, the autonomous-vehicle driving system may send instructions torelease the accelerator and apply the brakes.

An autonomous vehicle may be referred to as an autonomous car,driverless car, self-driving car, robotic car, or unmanned vehicle. Anautonomous vehicle may be a vehicle configured to sense its environmentand navigate or drive with little or no human input. For example, anautonomous vehicle may be configured to drive to any suitable locationand control or perform all safety-critical functions (e.g., driving,steering, braking, parking) for the entire trip, with the driver notexpected to control the vehicle at any time. As another example, anautonomous vehicle may allow a driver to safely turn their attentionaway from driving tasks in particular environments (e.g., on freeways),or an autonomous vehicle may provide control of a vehicle in all but afew environments, requiring little or no input or attention from thedriver.

An autonomous vehicle may be configured to drive with a driver presentin the vehicle, or an autonomous vehicle may be configured to operatethe vehicle with no driver present. As an example, an autonomous vehiclemay include a driver's seat with associated controls (e.g., steeringwheel, accelerator pedal, and brake pedal), and the vehicle may beconfigured to drive with no one seated in the driver's seat or withlittle or no input from a person seated in the driver's seat. As anotherexample, an autonomous vehicle may not include any driver's seat orassociated driver's controls, and the vehicle may perform substantiallyall driving functions (e.g., driving, steering, braking, parking, andnavigating) without human input. As another example, an autonomousvehicle may be configured to operate without a driver (e.g., the vehiclemay be configured to transport human passengers or cargo without adriver present in the vehicle). As another example, an autonomousvehicle may be configured to operate without any human passengers (e.g.,the vehicle may be configured for transportation of cargo without havingany human passengers onboard the vehicle).

In some implementations, a light source of a lidar system is locatedremotely from some of the other components of the lidar system such asthe scanner and the receiver. Moreover, a lidar system implemented in avehicle may include fewer light sources than scanners and receivers.

FIG. 8 illustrates an example configuration in which a laser-sensor link320 includes an optical link 330 and an electrical link 350 coupledbetween a laser 300 and a sensor 310. The laser 300 may be configured toemit pulses of light and may be referred to as a laser system, laserhead, or light source. The laser 300 may include, may be part of, may besimilar to, or may be substantially the same as the light source 110illustrated in FIG. 1 and discussed above. Further, the scanner 302, thereceiver 304, the controller 306, and the mirror 308 may be similar tothe scanner 120, the receiver 140, the controller 150, and the mirror115 discussed above. In the example of FIG. 8, the laser 300 is coupledto the remotely located sensor 310 by a laser-sensor link 320 (which maybe referred to as a link). The sensor 310 may be referred to as a sensorhead and may include the mirror 308, the scanner 302, the receiver 304,and the controller 306. In an example implementation, the laser 300includes a pulsed laser diode (e.g., a pulsed DFB laser) followed by anoptical amplifier, and light from the laser 300 is conveyed by anoptical fiber of the laser-sensor link 320 of a suitable length to thescanner 120 in a remotely located sensor 310.

The laser-sensor link 320 may include any suitable number of opticallinks 330 (e.g., 0, 1, 2, 3, 5, or 10) and any suitable number ofelectrical links 350 (e.g., 0, 1, 2, 3, 5, or 10). In the exampleconfiguration depicted in FIG. 8, the laser-sensor link 320 includes oneoptical link 330 from the laser 300 to an output collimator 340 and oneelectrical link 350 that connects the laser 300 to the controller 150.The optical link 330 may include optical fiber (which may be referred toas fiber-optic cable or fiber) that conveys, carries, transports, ortransmits light between the laser 300 and the sensor 310. The opticalfiber may be, for example, single-mode (SM) fiber, multi-mode (MM)fiber, large-mode-area (LMA) fiber, polarization-maintaining (PM) fiber,photonic-crystal or photonic-bandgap fiber, gain fiber (e.g.,rare-earth-doped optical fiber for use in an optical amplifier), or anysuitable combination thereof. The output collimator 340 receives opticalpulses conveyed from the laser 300 by the optical link 330 and producesa free-space optical beam 312 that includes the optical pulses. Theoutput collimator 340 directs the free-space optical beam 312 throughthe mirror 308 and to the scanner 302.

The electrical link 350 may include electrical wire or cable (e.g., acoaxial cable or twisted-pair cable) that conveys or transmitselectrical power and/or one or more electrical signals between the laser300 and the sensor 310. For example, the laser 300 may include a powersupply or a power conditioner that provides electrical power to thelaser 300, and additionally, the power supply or power conditioner mayprovide power to one or more components of the sensor 310 (e.g., thescanner 304, the receiver 304, and/or the controller 306) via the one ormore electrical links 350. The electrical link 350 in someimplementations may convey electrical signals that include data orinformation in analog or digital format. Further, the electrical link350 may provide an interlock signal from the sensor 310 to the laser300. If the controller 306 detects a fault condition indicating aproblem with the sensor 310 or the overall lidar system, the controller306 may change a voltage on the interlock line (e.g., from 5 V to 0 V)indicating that the laser 300 should shut down, stop emitting light, orreduce the power or energy of emitted light. A fault condition may betriggered by a failure of the scanner 302, a failure of the receiver304, or by a person or object coming within a threshold distance of thesensor 310 (e.g., within 0.1 m, 0.5 m, 1 m, 5 m, or any other suitabledistance).

As discussed above, a lidar system can include one or more processors todetermine a distance D to a target. In the implementation illustrated inFIG. 8, the controller 306 may be located in the laser 300 or in thesensor 310, or parts of the controller 150 may be distributed betweenthe laser 300 and the sensor 310. In an example implementation, eachsensor head 310 of a lidar system includes electronics (e.g., anelectronic filter, transimpedance amplifier, threshold detector, ortime-to-digital (TDC) converter) configured to receive or process asignal from the receiver 304 or from an APD or SPAD of the receiver 304.Additionally, the laser 300 may include processing electronicsconfigured to determine a time-of-flight value or a distance to thetarget based on a signal received from the sensor head 310 via theelectrical link 350.

Next, FIG. 9 illustrates an example vehicle 354 with a lidar system 351that includes a laser 352 with multiple sensor heads 360 coupled to thelaser 352 via multiple laser-sensor links 370. The laser 352 and thesensor heads 360 may be similar to the laser 300 and the sensor 310discussed above, in some implementations. For example, each of thelaser-sensor links 370 may include one or more optical links and/or oneor more electrical links. The sensor heads 360 in FIG. 9 are positionedor oriented to provide a greater than 30-degree view of an environmentaround the vehicle. More generally, a lidar system with multiple sensorheads may provide a horizontal field of regard around a vehicle ofapproximately 30°, 45°, 60°, 90°, 120°, 180°, 270°, or 360°. Each of thesensor heads may be attached to or incorporated into a bumper, fender,grill, side panel, spoiler, roof, headlight assembly, taillightassembly, rear-view mirror assembly, hood, trunk, window, or any othersuitable part of the vehicle.

In the example of FIG. 9, four sensor heads 360 are positioned at ornear the four corners of the vehicle (e.g., the sensor heads may beincorporated into a light assembly, side panel, bumper, or fender), andthe laser 352 may be located within the vehicle (e.g., in or near thetrunk). The four sensor heads 360 may each provide a 90° to 120°horizontal field of regard (FOR), and the four sensor heads 360 may beoriented so that together they provide a complete 360-degree view aroundthe vehicle. As another example, the lidar system 351 may include sixsensor heads 360 positioned on or around a vehicle, where each of thesensor heads 360 provides a 60° to 90° horizontal FOR. As anotherexample, the lidar system 351 may include eight sensor heads 360, andeach of the sensor heads 360 may provide a 45° to 60° horizontal FOR. Asyet another example, the lidar system 351 may include six sensor heads360, where each of the sensor heads 360 provides a 70° horizontal FORwith an overlap between adjacent FORs of approximately 10°. As anotherexample, the lidar system 351 may include two sensor heads 360 whichtogether provide a forward-facing horizontal FOR of greater than orequal to 30°.

Data from each of the sensor heads 360 may be combined or stitchedtogether to generate a point cloud that covers a greater than or equalto 30-degree horizontal view around a vehicle. For example, the laser352 may include a controller or processor that receives data from eachof the sensor heads 360 (e.g., via a corresponding electrical link 370)and processes the received data to construct a point cloud covering a360-degree horizontal view around a vehicle or to determine distances toone or more targets. The point cloud or information from the point cloudmay be provided to a vehicle controller 372 via a correspondingelectrical, optical, or radio link 370. In some implementations, thepoint cloud is generated by combining data from each of the multiplesensor heads 360 at a controller included within the laser 352 andprovided to the vehicle controller 372. In other implementations, eachof the sensor heads 360 includes a controller or processor thatconstructs a point cloud for a portion of the 360-degree horizontal viewaround the vehicle and provides the respective point cloud to thevehicle controller 372. The vehicle controller 372 then combines orstitches together the point clouds from the respective sensor heads 360to construct a combined point cloud covering a 360-degree horizontalview. Still further, the vehicle controller 372 in some implementationscommunicates with a remote server to process point cloud data.

In any event, the vehicle 354 may be an autonomous vehicle where thevehicle controller 372 provides control signals to various components390 within the vehicle 354 to maneuver and otherwise control operationof the vehicle 354. The components 390 are depicted in an expanded viewin FIG. 9 for ease of illustration only. The components 390 may includean accelerator 374, brakes 376, a vehicle engine 378, a steeringmechanism 380, lights 382 such as brake lights, head lights, reverselights, emergency lights, etc., a gear selector 384, and/or othersuitable components that effectuate and control movement of the vehicle354. The gear selector 384 may include the park, reverse, neutral, drivegears, etc. Each of the components 390 may include an interface viawhich the component receives commands from the vehicle controller 372such as “increase speed,” “decrease speed,” “turn left 5 degrees,”“activate left turn signal,” etc. and, in some cases, provides feedbackto the vehicle controller 372.

In some implementations, the vehicle controller 372 receives point clouddata from the laser 352 or sensor heads 360 via the link 370 andanalyzes the received point cloud data to sense or identify targets 130and their respective locations, distances, speeds, shapes, sizes, typeof target (e.g., vehicle, human, tree, animal), etc. The vehiclecontroller 372 then provides control signals via the link 370 to thecomponents 390 to control operation of the vehicle based on the analyzedinformation. For example, the vehicle controller 372 may identify anintersection based on the point cloud data and determine that theintersection is the appropriate location at which to make a left turn.Accordingly, the vehicle controller 372 may provide control signals tothe steering mechanism 380, the accelerator 374, and brakes 376 formaking a proper left turn. In another example, the vehicle controller372 may identify a traffic light based on the point cloud data anddetermine that the vehicle 354 needs to come to a stop. As a result, thevehicle controller 372 may provide control signals to release theaccelerator 374 and apply the brakes 376.

Example Receiver Implementation

FIG. 10 illustrates an example InGaAs avalanche photodiode (APD) 400.Referring back to FIG. 1, the receiver 140 may include one or more APDs400 configured to receive and detect light from input light such as thebeam 135. More generally, the APD 400 can operate in any suitablereceiver of input light. The APD 400 may be configured to detect aportion of pulses of light which are scattered by a target locateddownrange from the lidar system in which the APD 400 operates. Forexample, the APD 400 may receive a portion of a pulse of light scatteredby the target 130 depicted in FIG. 1, and generate an electrical-currentsignal corresponding to the received pulse of light.

The APD 400 may include doped or undoped layers of any suitablesemiconductor material, such as for example, silicon, germanium, InGaAs,InGaAsP, or indium phosphide (InP). Additionally, the APD 400 mayinclude an upper electrode 402 and a lower electrode 406 for couplingthe ADP 400 to an electrical circuit. The APD 400 for example may beelectrically coupled to a voltage source that supplies a reverse-biasvoltage V to the APD 400. Additionally, the APD 400 may be electricallycoupled to a transimpedance amplifier which receives electrical currentgenerated by the APD 400 and produces an output voltage signal thatcorresponds to the received current. The upper electrode 402 or lowerelectrode 406 may include any suitable electrically conductive material,such as for example a metal (e.g., gold, copper, silver, or aluminum), atransparent conductive oxide (e.g., indium tin oxide), a carbon-nanotubematerial, or polysilicon. In some implementations, the upper electrode402 is partially transparent or has an opening to allow input light 410to pass through to the active region of the APD 400. In FIG. 10, theupper electrode 402 may have a ring shape that at least partiallysurrounds the active region of the APD 400, where the active regionrefers to an area over which the APD 400 may receive and detect theinput light 410. The active region may have any suitable size ordiameter d, such as for example, a diameter of approximately 25 μm, 50μm, 80 μm, 100 μm, 200 μm, 500 μm, 1 mm, 2 mm, or 5 mm.

The APD 400 may include any suitable combination of any suitablesemiconductor layers having any suitable doping (e.g., n-doped, p-doped,or intrinsic undoped material). In the example of FIG. 10, the InGaAsAPD 400 includes a p-doped InP layer 420, an InP avalanche layer 422, anabsorption layer 424 with n-doped InGaAs or InGaAsP, and an n-doped InPsubstrate layer 426. Depending on the implementation, the APD 400 mayinclude separate absorption and avalanche layers, or a single layer mayact as both an absorption and avalanche region. The APD 400 may operateelectrically as a PN diode or a PIN diode, and, during operation, theAPD 400 may be reverse-biased with a positive voltage V applied to thelower electrode 406 with respect to the upper electrode 402. The appliedreverse-bias voltage V may have any suitable value, such as for exampleapproximately 5 V, 10 V, 20 V, 30 V, 50 V, 75 V, 100 V, or 200 V.

In FIG. 10, photons of the input light 410 may be absorbed primarily inthe absorption layer 424, resulting in the generation of electron-holepairs (which may be referred to as photo-generated carriers). Forexample, the absorption layer 424 may be configured to absorb photonscorresponding to the operating wavelength of the lidar system 100 (e.g.,any suitable wavelength between approximately 1400 nm and approximately1600 nm). In the avalanche layer 422, an avalanche-multiplicationprocess occurs where carriers (e.g., electrons or holes) generated inthe absorption layer 424 collide with the semiconductor lattice of theabsorption layer 424, and produce additional carriers through impactionization. This avalanche process can repeat numerous times so that onephoto-generated carrier may result in the generation of multiplecarriers. As an example, a single photon absorbed in the absorptionlayer 424 may lead to the generation of approximately 10, 50, 100, 200,500, 1000, 10,000, or any other suitable number of carriers through anavalanche-multiplication process. The carriers generated in an APD 400may produce an electrical current that is coupled to an electricalcircuit which may perform signal amplification, sampling, filtering,signal conditioning, analog-to-digital conversion, time-to-digitalconversion, pulse detection, threshold detection, rising-edge detection,or falling-edge detection.

The number of carriers generated from a single photo-generated carriermay increase as the applied reverse bias V is increased. If the appliedreverse bias V is increased above a particular value referred to as theAPD breakdown voltage, then a single carrier can trigger aself-sustaining avalanche process (e.g., the output of the APD 400 issaturated regardless of the input light level). The APD 400 that isoperated at or above a breakdown voltage may be referred to as asingle-photon avalanche diode (SPAD) and may be referred to as operatingin a Geiger mode or a photon-counting mode. The APD 400 that is operatedbelow a breakdown voltage may be referred to as a linear APD, and theoutput current generated by the APD 400 may be sent to an amplifiercircuit (e.g., a transimpedance amplifier). The receiver 140 (seeFIG. 1) may include an APD configured to operate as a SPAD and aquenching circuit configured to reduce a reverse-bias voltage applied tothe SPAD when an avalanche event occurs in the SPAD. The APD 400configured to operate as a SPAD may be coupled to an electronicquenching circuit that reduces the applied voltage V below the breakdownvoltage when an avalanche-detection event occurs. Reducing the appliedvoltage may halt the avalanche process, and the applied reverse-biasvoltage may then be re-set to await a subsequent avalanche event.Additionally, the APD 400 may be coupled to a circuit that generates anelectrical output pulse or edge when an avalanche event occurs.

In some implementations, the APD 400 or the APD 400 along withtransimpedance amplifier have a noise-equivalent power (NEP) that isless than or equal to 100 photons, 50 photons, 30 photons, 20 photons,or 10 photons. For example, the APD 400 may be operated as a SPAD andmay have a NEP of less than or equal to 20 photons. As another example,the APD 400 may be coupled to a transimpedance amplifier that producesan output voltage signal with a NEP of less than or equal to 50 photons.The NEP of the APD 400 is a metric that quantifies the sensitivity ofthe APD 400 in terms of a minimum signal (or a minimum number ofphotons) that the APD 400 can detect. The NEP may correspond to anoptical power (or to a number of photons) that results in asignal-to-noise ratio of 1, or the NEP may represent a threshold numberof photons above which an optical signal may be detected. For example,if the APD 400 has a NEP of 20 photons, then the input beam 410 with 20photons may be detected with a signal-to-noise ratio of approximately 1(e.g., the APD 400 may receive 20 photons from the input beam 410 andgenerate an electrical signal representing the input beam 410 that has asignal-to-noise ratio of approximately 1). Similarly, the input beam 410with 100 photons may be detected with a signal-to-noise ratio ofapproximately 5. In some implementations, the lidar system 100 with theAPD 400 (or a combination of the APD 400 and transimpedance amplifier)having a NEP of less than or equal to 100 photons, 50 photons, 30photons, 20 photons, or 10 photons offers improved detection sensitivitywith respect to a conventional lidar system that uses a PN or PINphotodiode. For example, an InGaAs PIN photodiode used in a conventionallidar system may have a NEP of approximately 10⁴ to 10⁵ photons, and thenoise level in a lidar system with an InGaAs PIN photodiode may be 10³to 10⁴ times greater than the noise level in a lidar system 100 with theInGaAs APD detector 400.

Referring back to FIG. 1, an optical filter may be located in front ofthe receiver 140 and configured to transmit light at one or moreoperating wavelengths of the light source 110 and attenuate light atsurrounding wavelengths. For example, an optical filter may be afree-space spectral filter located in front of APD 400 of FIG. 10. Thisspectral filter may transmit light at the operating wavelength of thelight source 110 (e.g., between approximately 1530 nm and 1560 nm) andattenuate light outside that wavelength range. As a more specificexample, light with wavelengths of approximately 400-1530 nm or1560-2000 nm may be attenuated by any suitable amount, such as forexample, by at least 5 dB, 10 dB, 20 dB, 30 dB, or 40 dB.

Next, FIG. 11 illustrates an APD 502 coupled to an examplepulse-detection circuit 504. The APD 502 can be similar to the APD 400discussed above with reference to FIG. 10, or can be any other suitabledetector. The pulse-detection circuit 504 can operate in the lidarsystem of FIG. 1 as part of the receiver 140. Further, thepulse-detection circuit 504 can operate in the receiver 164 of FIG. 2,the receiver 304 of FIG. 8, or any other suitable receiver. Thepulse-detection circuit 504 alternatively can be implemented in thecontroller 150, the controller 306, or another suitable controller. Insome implementations, parts of the pulse-detection circuit 504 canoperate in a receiver and other parts of the pulse-detection circuit 504can operate in a controller. For example, components 510 and 512 may bea part of the receiver 140, and components 514 and 516 may be a part ofthe controller 150.

The pulse-detection circuit 504 may include circuitry that receives asignal from a detector (e.g., an electrical current from the APD 502)and performs current-to-voltage conversion, signal amplification,sampling, filtering, signal conditioning, analog-to-digital conversion,time-to-digital conversion, pulse detection, threshold detection,rising-edge detection, or falling-edge detection. The pulse-detectioncircuit 504 may determine whether an optical pulse has been received bythe APD 502 or may determine a time associated with receipt of anoptical pulse by the APD 502. Additionally, the pulse-detection circuit504 may determine a duration of a received optical pulse. In an exampleimplementation, the pulse-detection circuit 504 includes atransimpedance amplifier (TIA) 510, a gain circuit 512, a comparator514, and a time-to-digital converter (TDC) 516.

The TIA 510 may be configured to receive an electrical-current signalfrom the APD 502 and produce a voltage signal that corresponds to thereceived electrical-current signal. For example, in response to areceived optical pulse, the APD 502 may produce a current pulsecorresponding to the optical pulse. The TIA 510 may receive the currentpulse from the APD 502 and produce a voltage pulse that corresponds tothe received current pulse. The TIA 510 may also act as an electronicfilter. For example, the TIA 510 may be configured as a low-pass filterthat removes or attenuates high-frequency electrical noise byattenuating signals above a particular frequency (e.g., above 1 MHz, 10MHz, 20 MHz, 50 MHz, 100 MHz, 200 MHz, or any other suitable frequency).

The gain circuit 512 may be configured to amplify a voltage signal. Asan example, the gain circuit 512 may include one or morevoltage-amplification stages that amplify a voltage signal received fromthe TIA 510. For example, the gain circuit 512 may receive a voltagepulse from the TIA 510, and the gain circuit 512 may amplify the voltagepulse by any suitable amount, such as for example, by a gain ofapproximately 3 dB, 10 dB, 20 dB, 30 dB, 40 dB, or 50 dB. Additionally,the gain circuit 512 may also act as an electronic filter configured toremove or attenuate electrical noise.

The comparator 514 may be configured to receive a voltage signal fromthe TIA 510 or the gain circuit 512 and produce an electrical-edgesignal (e.g., a rising edge or a falling edge) when the received voltagesignal rises above or falls below a particular threshold voltage V_(T).As an example, when a received voltage rises above V_(T), the comparator514 may produce a rising-edge digital-voltage signal (e.g., a signalthat steps from approximately 0 V to approximately 2.5 V, 3.3 V, 5 V, orany other suitable digital-high level). As another example, when areceived voltage falls below V_(T), the comparator 514 may produce afalling-edge digital-voltage signal (e.g., a signal that steps down fromapproximately 2.5 V, 3.3 V, 5 V, or any other suitable digital-highlevel to approximately 0 V). The voltage signal received by thecomparator 514 may be received from the TIA 510 or the gain circuit 512and may correspond to an electrical-current signal generated by the APD502. For example, the voltage signal received by the comparator 514 mayinclude a voltage pulse that corresponds to an electrical-current pulseproduced by the APD 502 in response to receiving an optical pulse. Thevoltage signal received by the comparator 514 may be an analog signal,and an electrical-edge signal produced by the comparator 514 may be adigital signal.

The time-to-digital converter (TDC) 516 may be configured to receive anelectrical-edge signal from the comparator 514 and determine an intervalof time between emission of a pulse of light by the light source andreceipt of the electrical-edge signal. The output of the TDC 516 may bea numerical value that corresponds to the time interval determined bythe TDC 516. In some implementations, the TDC 516 has an internalcounter or clock with any suitable period, such as for example, 5 ps, 10ps, 15 ps, 20 ps, 30 ps, 50 ps, 100 ps, 0.5 ns, 1 ns, 2 ns, 5 ns, or 10ns. The TDC 516 for example may have an internal counter or clock with a20 ps period, and the TDC 516 may determine that an interval of timebetween emission and receipt of a pulse is equal to 25,000 time periods,which corresponds to a time interval of approximately 0.5 microseconds.Referring back to FIG. 1, the TDC 516 may send the numerical value“25000” to a processor or controller 150 of the lidar system 100, whichmay include a processor configured to determine a distance from thelidar system 100 to the target 130 based at least in part on an intervalof time determined by a TDC 516. The processor may receive a numericalvalue (e.g., “25000”) from the TDC 516 and, based on the received value,the processor may determine the distance from the lidar system 100 to atarget 130.

Example Method for Detecting and Removing Debris on a Window in a LidarSystem

FIG. 12 depicts a flow diagram of an example method 600 for detectingand removing debris on a window in a lidar system. The method may beimplemented by various components of the lidar system 100 as shown inFIG. 1 including the light source 110, the scanner 120, the receiver140, and the controller 150. For ease of illustration only, some of thesteps of the method 600 may be described below with reference to aparticular component of the lidar system 100. However, each of themethod steps may be implemented by any suitable component in anysuitable manner. In some embodiments, the method or a portion thereofcan be implemented in a set of instructions stored on acomputer-readable memory and executable on one or more processors or thecontroller 150.

At block 602, light pulses are emitted by the light source 110. In someimplementations, the controller 150 directs the light source 110 to emitlight pulses by providing instructions, a control signal, or a triggersignal to the light source 110 indicating when the light source 110should produce optical pulses. The light pulses are then emitted withparticular characteristics, such as a particular pulse rate or pulserepetition frequency, a peak power, an average power, a pulse energy, apulse duration, a wavelength, etc.

At block 604, the emitted light pulses are directed, via the scanner120, at various scan angles or orientations relative to a forward-facingdirection of the vehicle. In this manner, the emitted light pulses arescanned across a horizontal FOR (e.g., from −60 degrees to +60 degreeswith respect to the forward-facing direction of the vehicle). In someimplementations, the controller 150 provides a drive signal to thescanner 120 for rotating the scanning mirror across a horizontal FOR todirect light pulses toward different points within the horizontal FOR.The emitted light pulses may also be directed, via the scanner 120across a vertical FOR (e.g., from −15 degrees vertical to +15 degreesvertical. In some implementations, the controller 150 provides a drivesignal to the scanner 120 for rotating the same scanning mirror oranother scanning mirror across a vertical FOR to direct light pulsestoward different points within the vertical FOR. For example, thescanner 120 may direct light pulses across a horizontal FOR at a firstvertical orientation (e.g., +15 degrees vertical) to generate a scanline. Then the scanner 120 may direct light pulses across the horizontalFOR at another vertical orientation (e.g., +14 degrees vertical) togenerate another scan line. A collection of information from returnlight pulses or pixels captured in succession across the FOR may bereferred to as a depth map, a point cloud, or a frame. The lidar system100 may be configured to repeatedly capture or generate point clouds ofa field of regard at any suitable frame rate between approximately 0.1frames per second (FPS) and approximately 1,000 FPS.

At block 606, light from some of the light pulses is scattered by remotetargets such as the target 130, as shown in FIG. 1 and detected by thereceiver 140, for example to identify return light pulses correspondingto the emitted light pulses. Characteristics of the return light pulsesare then used to generate a point cloud having respective pixels. Thecharacteristics of the return light pulses may also be used to detectdebris on a window 157 of the lidar system 100 having an exterior-facingsurface arranged in front of the light source 110, scanner 120, andreceiver 140 (block 608).

More specifically, debris on the window 157 may be detected when returnlight pulses are distorted or smeared out due to partial scattering ofan emitted light pulse off “soft” targets such as the rain. Accordingly,the receiver 140 may detect a tailing effect in a return light pulseindicative of attenuation or distortion of the return light pulse. Morespecifically, a return light pulse may be detected by the APD 502 andthe pulse-detection circuit 504 as shown in FIG. 11 and analyzed by thepulse-detection circuit 504 to convert an optical signal for the returnlight pulse to an electrical voltage signal and to identifycharacteristics of the return light pulse.

Characteristics of the return light pulse may include a rise time forthe return light pulse indicating the duration of the rising edge forthe return light pulse, a fall time indicating the duration of thefalling edge for the return light pulse, a pulse duration for the returnlight pulse, etc. The characteristics of the return light pulse may beprovided to the controller 150 (e.g., by the TDCs 516) and thecontroller 150 may analyze the characteristics of the return light pulseto detect rain. The controller 150 may then detect a tailing effect whenthe rise time, fall time, or pulse duration exceeds a threshold time.The tailing effect may be interpreted as rain. The controller 150 maythen determine that there is rain or other debris on the window 157 inresponse to detecting rain in the area.

Debris on the window 157 may also be detected via a rain or otherweather sensor within the lidar system 100. In some implementations, therain sensor detects rain by transmitting optical pulses at the window157. When the window is wet, at least some of the optical pulses are notscattered and then received by the rain sensor.

In other implementations, debris on the window 157 may be determined bycommunicating with a GPS within the vehicle 354 to obtain the currentlocation of the vehicle 354 and a weather data service to obtain weatherdata at the current location. For example, the controller 150 maycommunicate with the GPS and in response to obtaining the currentlocation of the vehicle 354, the controller 150 may transmit a requestto the National Weather Service for weather data at the currentlocation. The controller 150 may then determine that there is rain orother debris on the window 157 in response to receiving an indicationfrom the National Weather Service that it is raining at the currentlocation.

In yet other implementations, debris may be detected when the receiverdoes not detect return light pulses from the same receiver field of viewover a threshold period of time or number of frames. As mentioned above,the lidar system 100 may be configured to repeatedly capture or generatepoint clouds of a field of regard at any suitable frame rate. The lidarsystem 100 collects a pixel from the same receiver field of view (e.g.,−7° horizontal and +3° vertical) at the same orientation with respect tothe forward-facing direction of the vehicle 354 during each frame forseveral frames. When a return light pulse is not received from the samereceiver field of view over a threshold number of frames (e.g., 100frames) and/or when the vehicle 354 is moving, such that the lidarsystem 100 is capturing images at different locations, the controller150 may determine that the receiver field of view is blocked.Accordingly, the controller 150 determines that there is debris on aportion of the window 157 corresponding to the blocked receiver field ofview.

In response to detecting debris on the window 157, one or moreultrasonic transducers 159 a, 159 b vibrate at an ultrasonic frequency(e.g., 20 kHz, 50 kHz, 100 kHz, 1 MHz, 10 MHz, etc.) to remove debris,such as water, fog, dirt, etc. from the exterior surface (surface B) ofthe window 157 (block 610). In some implementations, a single ultrasonictransducer 159 a is used to generate a standing ultrasonic wave and thevibration causes the debris to bounce off the window 157. In otherimplementations, two ultrasonic transducers 159 a, 159 b vibrate with aparticular phase shift to generate a travelling ultrasonic wave. Thetravelling ultrasonic wave causes the debris to sweep across theexterior surface of the window 157.

The oleophobic and/or hydrophobic coating on the exterior surface of thewindow 157 may also be used in combination with the ultrasonictransducers 159 a, 159 b to remove debris. Additionally, windshieldwipers and/or a nozzle that sprays water or cleaner may be attached tothe exterior surface of the window 157. When debris is detected at thewindow 157, any suitable combination of the windshield wipers, spray,hydrophobic or oleophobic layer, and vibration may be used to remove thedebris.

In some implementations, the controller 150 is communicatively coupledto the ultrasonic transducers 159 a, 159 b, windshield wipers, and/ornozzle. When the controller 150 detects debris on the exterior surfaceof the window or receives an indication of debris (e.g., from the rainor weather sensor), the controller 150 provides control signalsactivating the ultrasonic transducers 159 a, 159 b, windshield wipers,and/or nozzle. In some scenarios, the controller 150 adjusts thefrequency intensity of the vibration including the frequency and/oramplitude of the ultrasonic waves according to the amount, size, or typeof debris on the window. For example, the frequency may be 20 kHz whenthere are a few droplets of rain on the window. When the entire windowis covered with debris, the frequency may be set to 10 MHz. The amount,size, or type of debris may be determined by the severity of a storm forexample, as indicated by the National Weather Service. Additionally, theamount, size, or type of debris may be determined based on the number ofreceiver fields of view that are blocked. For example, the frequency maybe 20 kHz when less than a threshold number of receiver fields of view(e.g., 3) are identified as blocked. When at least the threshold numberof receiver fields of view are identified as blocked, the frequency maybe increased to 10 MHz and/or the amplitude of the ultrasonic wave maybe increased.

Also in some implementations, the controller 150 provides controlsignals activating the ultrasonic transducers 159 a, 159 b for aparticular time period (e.g., 10 seconds), periodically for a particularnumber of time periods (e.g., for 10 seconds out of every 30 secondsover the course of 10 minutes), and/or until the debris is no longerdetected. For example, the ultrasonic transducers 159 a, 159 b may beactivated until the controller 150 receives an indication from theNational Weather Service that the rain has stopped, the rain, otherweather sensor, or controller 150 stops detecting rain, or thecontroller 150 no longer detects a blockage at a particular receiverfield of view based on the return light pulses at the receiver field ofview. When the controller 150 determines that the debris has beenremoved or the rain has subsided, the controller 150 may provide controlsignals to deactivate the ultrasonic transducers 159 a, 159 b.

Example Method for Verifying Removal of Debris from a Window in a LidarSystem

FIG. 13 depicts a flow diagram of an example method 700 for verifyingthat debris has been removed from a window of a lidar system. The methodmay be implemented by various components of the lidar system 100 asshown in FIG. 1 including the light source 110, the scanner 120, thereceiver 140, and the controller 150. For ease of illustration only,some of the steps of the method 700 may be described below withreference to a particular component of the lidar system 100. However,each of the method steps may be implemented by any suitable component inany suitable manner. In some embodiments, the method or a portionthereof can be implemented in a set of instructions stored on acomputer-readable memory and executable on one or more processors or thecontroller 150.

At block 702, debris may be detected on a window 157 of the lidar system100 having an exterior-facing surface arranged in front of the lightsource 110, scanner 120, and receiver 140. More specifically, debris onthe window 157 may be detected when return light pulses are distorted orsmeared out due to partial scattering of an emitted light pulse off“soft” targets such as the rain. Accordingly, the receiver 140 maydetect a tailing effect in a return light pulse indicative ofattenuation or distortion of the return light pulse. The tailing effectmay be interpreted as rain. The controller 150 may then determine thatthere is rain or other debris on the window 157 in response to detectingrain in the area.

Debris on the window 157 may also be detected via a rain or otherweather sensor within the lidar system 100. In some implementations, therain sensor detects rain by transmitting optical pulses at the window157. When the window is wet, at least some of the optical pulses are notscattered and then received by the rain sensor.

In other implementations, debris on the window 157 may be determined bycommunicating with a GPS within the vehicle 354 to obtain the currentlocation of the vehicle 354 and a weather data service to obtain weatherdata at the current location. For example, the controller 150 maycommunicate with the GPS and in response to obtaining the currentlocation of the vehicle 354, the controller 150 may transmit a requestto the National Weather Service for weather data at the currentlocation. The controller 150 may then determine that there is rain orother debris on the window 157 in response to receiving an indicationfrom the National Weather Service that it is raining at the currentlocation.

In yet other implementations, debris may be detected when the receiverdoes not detect return light pulses from the same receiver field of viewover a threshold period of time or number of frames. As mentioned above,the lidar system 100 may be configured to repeatedly capture or generatepoint clouds of a field of regard at any suitable frame rate. The lidarsystem 100 collects a pixel from the same receiver field of view (e.g.,−7° horizontal and +3° vertical) at the same orientation with respect tothe forward-facing direction of the vehicle 354 during each frame forseveral frames. When a return light pulse is not received from the samereceiver field of view over a threshold number of frames (e.g., 100frames) and/or when the vehicle 354 is moving, such that the lidarsystem 100 is capturing images at different locations, the controller150 may determine that the receiver field of view is blocked.Accordingly, the controller 150 determines that there is debris on aportion of the window 157 corresponding to the blocked receiver field ofview.

The type of debris may then be determined based on the manner in whichthe debris is detected (block 704). For example, when debris is detectedfrom a rain sensor or by communicating with a National Weather Serviceto determine that it is raining, the controller 150 may determine thedebris is a persistent or recurring type (rain). On the other hand, whendebris is detected based on one or several blocked receiver fields ofview, the controller 150 may determine that the debris is dirt oranother substance and the debris is an isolated occurrence.

Then at block 706, a remedial action may be initiated to remove thedebris. For example, in response to detecting debris on the window 157,one or more ultrasonic transducers 159 a, 159 b vibrate at an ultrasonicfrequency to remove the debris. The oleophobic and/or hydrophobiccoating on the exterior surface of the window 157 may also be used incombination with the ultrasonic transducers 159 a, 159 b to removedebris. Additionally, windshield wipers and/or a nozzle that sprayswater or cleaner may be attached to the exterior surface of the window157. When debris is detected at the window 157, any suitable combinationof the windshield wipers, spray, hydrophobic or oleophobic layer, andvibration may be used to remove the debris.

If the debris is a persistent type such as rain (block 708), the lidarsystem 100 may continue taking remedial actions (block 706) until therain subsides for example. On other hand, if the debris is an isolatedoccurrence such as dirt or another substance, the lidar system 100 maydetermine once again whether there is debris on the window 157 (block710). If the debris has successfully been removed from the window 157,the process ends. However, if the debris remains on the window 157, thelidar system 100 may generate one or more indications or alerts (block712) for the operator, passenger, or driver of the vehicle to indicatethat debris is likely present on the surface of the window. For example,the indications or alerts may be displayed on a head unit of thevehicle.

Example Method for Adjusting Wave Characteristics

FIG. 14 depicts a flow diagram of an example method 800 for adjustingcharacteristics of a wave created by ultrasonic transducers attached toa window in a lidar system based on the amount of debris on the window.The method may be implemented by various components of the lidar system100 as shown in FIG. 1 including the light source 110, the scanner 120,the receiver 140, and the controller 150. For ease of illustration only,some of the steps of the method 800 may be described below withreference to a particular component of the lidar system 100. However,each of the method steps may be implemented by any suitable component inany suitable manner. In some embodiments, the method or a portionthereof can be implemented in a set of instructions stored on acomputer-readable memory and executable on one or more processors or thecontroller 150.

At block 802, the lidar system 100 may determine that a receiver fieldof view (FOV) is blocked. For example, the lidar system 100 may beconfigured to repeatedly capture or generate point clouds of a field ofregard at any suitable frame rate. The lidar system 100 collects a pixelfrom the same receiver field of view (e.g., −7° horizontal and +3°vertical) at the same orientation with respect to the forward-facingdirection of the vehicle 354 during each frame for several frames. Whena return light pulse is not received from the same receiver field ofview over a threshold number of frames (e.g., 100 frames) and/or whenthe vehicle 354 is moving, such that the lidar system 100 is capturingimages at different locations, the controller 150 may determine that thereceiver field of view is blocked. Accordingly, the controller 150determines that there is debris on a portion of the window 157corresponding to the blocked receiver field of view (block 804).

In some scenarios, several receiver fields of view may be blocked forexample, when there is a large piece of dirt or other substance on thewindow 157. The number of blocked receiver fields of view may bedetermined (block 806) by aggregating each receiver field of view wherea return light pulse is not received over a threshold number of frames(e.g., 100 frames).

The amount or size of the debris may be determined based on the numberof blocked receiver fields of view (block 808). For example, when thereare two blocked receiver fields of view, the size of the debris may besmaller than when there are nine blocked receiver fields of view. Theamount or size of the debris may then be compared to a threshold amountor size (block 810). When the amount or size is at or below thethreshold amount or size, one or more ultrasonic transducers 159 a, 159b may create a first wave at an ultrasonic frequency having a firstintensity (block 812). The controller 150 may provide control signalsactivating the ultrasonic transducers 159, 159 b and/or indicatingcharacteristics of the first wave, such as the amplitude and frequency.

When the amount or size is greater than the threshold amount or size,the ultrasonic transducers 159 a, 159 b may create a second wave at anultrasonic frequency having a second intensity which is greater than thefirst intensity (block 814). The controller 150 may provide controlsignals activating the ultrasonic transducers 159, 159 b and/orindicating characteristics of the second wave, such as the amplitude andfrequency. For example, the amplitude and/or frequency for the secondwave may be increased relative to the amplitude and/or frequency for thefirst wave. More specifically, the frequency of the first wave may be 20kHz while the frequency of the second wave may be 10 MHz. While themethod 800 includes waves created at two different intensity levelsbased on the amount or size of the debris, waves may be created at anysuitable number of intensity levels. For example, the intensity level ofthe wave may be in proportion to the amount or size of the debris or thelidar system 100 may identify several size ranges each corresponding todifferent intensity levels. In yet other implementations, the controller150 may provide control signals to increase the intensity of the wavequadratically, polynomially, or exponentially in accordance with theamount or size of the debris.

General Considerations

In some cases, a computing device may be used to implement variousmodules, circuits, systems, methods, or algorithm steps disclosedherein. As an example, all or part of a module, circuit, system, method,or algorithm disclosed herein may be implemented or performed by ageneral-purpose single- or multi-chip processor, a digital signalprocessor (DSP), an ASIC, a FPGA, any other suitable programmable-logicdevice, discrete gate or transistor logic, discrete hardware components,or any suitable combination thereof. A general-purpose processor may bea microprocessor, or, any conventional processor, controller,microcontroller, or state machine. A processor may also be implementedas a combination of computing devices, e.g., a combination of a DSP anda microprocessor, a plurality of microprocessors, one or moremicroprocessors in conjunction with a DSP core, or any other suchconfiguration.

In particular embodiments, one or more implementations of the subjectmatter described herein may be implemented as one or more computerprograms (e.g., one or more modules of computer-program instructionsencoded or stored on a computer-readable non-transitory storage medium).As an example, the steps of a method or algorithm disclosed herein maybe implemented in a processor-executable software module which mayreside on a computer-readable non-transitory storage medium. Inparticular embodiments, a computer-readable non-transitory storagemedium may include any suitable storage medium that may be used to storeor transfer computer software and that may be accessed by a computersystem. Herein, a computer-readable non-transitory storage medium ormedia may include one or more semiconductor-based or other integratedcircuits (ICs) (such, as for example, field-programmable gate arrays(FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs),hybrid hard drives (HHDs), optical discs (e.g., compact discs (CDs),CD-ROM, digital versatile discs (DVDs), blue-ray discs, or laser discs),optical disc drives (ODDs), magneto-optical discs, magneto-opticaldrives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes,flash memories, solid-state drives (SSDs), RAM, RAM-drives, ROM, SECUREDIGITAL cards or drives, any other suitable computer-readablenon-transitory storage media, or any suitable combination of two or moreof these, where appropriate. A computer-readable non-transitory storagemedium may be volatile, non-volatile, or a combination of volatile andnon-volatile, where appropriate.

In some cases, certain features described herein in the context ofseparate implementations may also be combined and implemented in asingle implementation. Conversely, various features that are describedin the context of a single implementation may also be implemented inmultiple implementations separately or in any suitable sub-combination.Moreover, although features may be described above as acting in certaincombinations and even initially claimed as such, one or more featuresfrom a claimed combination may in some cases be excised from thecombination, and the claimed combination may be directed to asub-combination or variation of a sub-combination.

While operations may be depicted in the drawings as occurring in aparticular order, this should not be understood as requiring that suchoperations be performed in the particular order shown or in sequentialorder, or that all operations be performed. Further, the drawings mayschematically depict one more example processes or methods in the formof a flow diagram or a sequence diagram. However, other operations thatare not depicted may be incorporated in the example processes or methodsthat are schematically illustrated. For example, one or more additionaloperations may be performed before, after, simultaneously with, orbetween any of the illustrated operations. Moreover, one or moreoperations depicted in a diagram may be repeated, where appropriate.Additionally, operations depicted in a diagram may be performed in anysuitable order. Furthermore, although particular components, devices, orsystems are described herein as carrying out particular operations, anysuitable combination of any suitable components, devices, or systems maybe used to carry out any suitable operation or combination ofoperations. In certain circumstances, multitasking or parallelprocessing operations may be performed. Moreover, the separation ofvarious system components in the implementations described herein shouldnot be understood as requiring such separation in all implementations,and it should be understood that the described program components andsystems may be integrated together in a single software product orpackaged into multiple software products.

Various implementations have been described in connection with theaccompanying drawings. However, it should be understood that the figuresmay not necessarily be drawn to scale. As an example, distances orangles depicted in the figures are illustrative and may not necessarilybear an exact relationship to actual dimensions or layout of the devicesillustrated.

The scope of this disclosure encompasses all changes, substitutions,variations, alterations, and modifications to the example embodimentsdescribed or illustrated herein that a person having ordinary skill inthe art would comprehend. The scope of this disclosure is not limited tothe example embodiments described or illustrated herein. Moreover,although this disclosure describes or illustrates respective embodimentsherein as including particular components, elements, functions,operations, or steps, any of these embodiments may include anycombination or permutation of any of the components, elements,functions, operations, or steps described or illustrated anywhere hereinthat a person having ordinary skill in the art would comprehend.

The term “or” as used herein is to be interpreted as an inclusive ormeaning any one or any combination, unless expressly indicated otherwiseor indicated otherwise by context. Therefore, herein, the expression “Aor B” means “A, B, or both A and B.” As another example, herein, “A, Bor C” means at least one of the following: A; B; C; A and B; A and C; Band C; A, B and C. An exception to this definition will occur if acombination of elements, devices, steps, or operations is in some wayinherently mutually exclusive.

As used herein, words of approximation such as, without limitation,“approximately, “substantially,” or “about” refer to a condition thatwhen so modified is understood to not necessarily be absolute or perfectbut would be considered close enough to those of ordinary skill in theart to warrant designating the condition as being present. The extent towhich the description may vary will depend on how great a change can beinstituted and still have one of ordinary skill in the art recognize themodified feature as having the required characteristics or capabilitiesof the unmodified feature. In general, but subject to the precedingdiscussion, a numerical value herein that is modified by a word ofapproximation such as “approximately” may vary from the stated value by±0.5%, ±1%, ±2%, ±3%, ±4%, ±5%, ±10%, ±12%, or ±15%.

As used herein, the terms “first,” “second,” “third,” etc. may be usedas labels for nouns that they precede, and these terms may notnecessarily imply a particular ordering (e.g., a particular spatial,temporal, or logical ordering). As an example, a system may be describedas determining a “first result” and a “second result,” and the terms“first” and “second” may not necessarily imply that the first result isdetermined before the second result.

As used herein, the terms “based on” and “based at least in part on” maybe used to describe or present one or more factors that affect adetermination, and these terms may not exclude additional factors thatmay affect a determination. A determination may be based solely on thosefactors which are presented or may be based at least in part on thosefactors. The phrase “determine A based on B” indicates that B is afactor that affects the determination of A. In some instances, otherfactors may also contribute to the determination of A. In otherinstances, A may be determined based solely on B.

What is claimed is:
 1. A lidar system comprising: a light sourceconfigured to emit light pulses; a scanner configured to scan a field ofview of the light source across a field of regard of the lidar system,including direct the light pulses at different angles relative to aforward-facing direction toward different points within the field ofregard; a receiver configured to detect light from some of the lightpulses scattered by one or more remote targets to generate respectivepixels; a window having an exterior-facing surface, the window arrangedin front of the light source, the scanner, and the receiver, wherein theemitted light pulses are transmitted through the window toward the oneor more remote targets, and light from the some light pulses isscattered by the one or more remote targets, transmitted through thewindow, and detected at the receiver; and one or more ultrasonictransducers attached to the window and configured to cause vibration atthe window to remove debris from the exterior-facing surface of thewindow.
 2. The lidar system of claim 1, further comprising: a controllerconfigured to detect the debris on the window and transmit a controlsignal to the one or more ultrasonic transducers to vibrate.
 3. Thelidar system of claim 2, wherein the controller detects the debris onthe window by receiving an indication of debris from a rain sensorwithin a vehicle in which the lidar system operates.
 4. The lidar systemof claim 2, wherein to detect the debris on the window the controller isconfigured to: receive an indication from the receiver of at least aportion of a duration of a return light pulse; compare the indication ofthe at least a portion of the duration to a threshold duration; and inresponse to determining that the at least a portion of the durationexceeds the threshold duration, identify debris on the window.
 5. Thelidar system of claim 2, wherein to detect the debris on the window thecontroller is configured to: receive, from a positioning sensor, anindication of the geolocation of a vehicle in which the lidar systemoperates; transmit a request to a weather data service for currentweather data for the received geolocation; receive, from the weatherdata service, a response to the request indicating rain; and detectdebris on the window based on the indication of rain.
 6. The lidarsystem of claim 2, wherein to detect the debris on the window thecontroller is configured to: not receive an indication from the receiverof a return light pulse corresponding to a same angle relative to theforward-facing direction of a vehicle in which the lidar system operatesfor more than a threshold number of frames while the vehicle is moving;and detect debris on the window at the angle relative to theforward-facing direction of the vehicle with respect to the lightsource.
 7. The lidar system of claim 2, wherein the controller isfurther configured to: detect a size or amount of the debris on thewindow; and transmit a control signal to the one or more ultrasonictransducers indicating an intensity at which to vibrate, wherein theintensity is selected based on the size or amount of the debris.
 8. Thelidar system of claim 1, wherein the exterior-facing surface of thewindow is provided with a hydrophobic or oleophobic layer.
 9. The lidarsystem of claim 1, wherein the one or more ultrasonic transducersinclude a plurality of ultrasonic transducers vibrating with aparticular phase shift to generate a travelling ultrasonic wave, therebycausing the debris to sweep across the exterior-facing surface of thewindow.
 10. The lidar system of claim 1, wherein: the window is anexternal window; and the lidar system further comprises an internalwindow arranged between the external window and the light source,scanner, and receiver.
 11. The lidar system of claim 10, wherein theinternal window is configured to provide an airtight or watertight sealfor the lidar system.
 12. A method for removing debris from a window ofa lidar system, the method comprising: emitting light pulses by a lightsource in a lidar system; scanning, by a scanner in the lidar system, afield of view of the light source across a field of regard of the lidarsystem, including directing the light pulses toward different pointswithin the field of regard; detecting, by a receiver in the lidarsystem, light from some of the light pulses scattered by one or moreremote targets to generate respective pixels; detecting debris on awindow having an exterior-facing surface arranged in front of the lightsource, the scanner, and the receiver, wherein the emitted light pulsesare transmitted through the window toward the one or more remote targetsand light from the some light pulses is scattered by the one or moreremote targets, transmitted through the window, and detected at thereceiver; and causing one or more ultrasonic transducers attached to thewindow to vibrate in response to detecting the debris on the window toremove the debris from the exterior-facing surface of the window. 13.The method of claim 12, wherein detecting the debris on the windowincludes receiving an indication of debris from a rain sensor within avehicle in which the lidar system operates.
 14. The method of claim 12,wherein detecting the debris on the window includes: receiving anindication from the receiver of at least a portion of a duration of areturn light pulse; comparing the indication of the at least a portionof the duration to a threshold duration; and in response to determiningthat the at least a portion of the duration exceeds the thresholdduration, identifying debris on the window.
 15. The method of claim 12,wherein detecting the debris on the window includes: receiving, from apositioning sensor, an indication of the geolocation of a vehicle inwhich the lidar system operates; transmitting a request to a weatherdata service for current weather data for the received geolocation;receiving, from the weather data service, a response to the requestindicating rain; and detecting debris on the window based on theindication of rain.
 16. The method of claim 12, wherein detecting thedebris on the window includes: not receiving an indication from thereceiver of a return light pulse corresponding to a same angle relativeto a forward-facing direction of a vehicle in which the lidar systemoperates for more than a threshold number of frames while the vehicle ismoving; and detecting debris on the window at the angle relative to theforward-facing direction of the vehicle with respect to the lightsource.
 17. The method of claim 12, further comprising: detecting a sizeor amount of the debris on the window; and causing the one or moreultrasonic transducers to vibrate at a particular intensity selectedbased on the size or amount of the debris.
 18. A controller in a lidarsystem comprising: one or more processors; and a non-transitorycomputer-readable memory coupled to the one or more processors andstoring instructions thereon that, when executed by the one or moreprocessors, cause the controller to: provide a control signal to a lightsource to emit light pulses; detect debris on a window having anexterior-facing surface arranged in front of the light source, ascanner, and a receiver, wherein the emitted light pulses aretransmitted through the window toward one or more remote targets andlight from the some light pulses is scattered by the one or more remotetargets, transmitted through the window, and detected at the receiver;and provide a control signal to one or more ultrasonic transducersattached to the window to vibrate to remove debris from theexterior-facing surface of the window.
 19. The controller of claim 18,wherein to detect the debris on the window the instructions cause thecontroller to receive an indication of debris from a rain sensor withina vehicle in which the lidar system operates.
 20. The controller ofclaim 18, wherein to detect the debris on the window the instructionscause the controller to: not receive an indication from the receiver ofa return light pulse corresponding to a same angle relative to aforward-facing direction of a vehicle in which the lidar system operatesfor more than a threshold number of frames while the vehicle is moving;and detect debris on the window at the angle relative to theforward-facing direction of the vehicle with respect to the lightsource.
 21. The controller of claim 18, wherein to detect the debris onthe window the instructions cause the controller to: receive anindication from the receiver of at least a portion of a duration of areturn light pulse; compare the indication of the at least a portion ofthe duration to a threshold duration; and in response to determiningthat the at least a portion of the duration exceeds the thresholdduration, identify debris on the window.
 22. The controller of claim 18,wherein the instructions further cause the controller to: detect a sizeor amount of the debris on the window; and provide a control signal tothe one or more ultrasonic transducers indicating an intensity at whichto vibrate, wherein the intensity is selected based on the size oramount of the debris.